{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "66f759da",
   "metadata": {},
   "source": [
    "### 3.使用迭代方法实现最小二乘\n",
    "\n",
    "最简单的程序是我们直接计算出最佳参数的公式然后计算即可，但是作为其他机器学习方法的一个基础，当我们建立的模型比较复杂、或者没有解析的方法可以求解参数的时候(比如在飞行动力学的性能计算中常会有的形如 $x^4+ax+b=0$ 的方程，他并没有解析解），这个时候我们更常用迭代的方式来求解模型参数\n",
    "\n",
    "**下面我们介绍一种常用的迭代方法——梯度下降法：**\n",
    "\n",
    "在求解机器学习参数的优化算法中，基于梯度下降的优化算法(Gradient Descent, GD)使用较多。**这种方法的原理实际上就是，**函数在某一点沿着梯度方向变化率是最大的，假如我们将函数比作一座山，那么我们要寻找最小值就是要找到最低的地方，也就是山脚，当我们并不能看清这座山的全貌时，我们就可以以使用梯度下降的方法，沿着山最陡峭的地方一路往下走，每走一段我们就重新评估一次，那么我们最终可以成功抵达山谷\n",
    "\n",
    "当然了，这种方法也会有一定的错误存在，可以看到如图所示的两个箭头指向的地方实际上都是低处，但是由于我们出发的路径选择，最终导致了我们选择了较高的一点"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "3f038783",
   "metadata": {},
   "source": [
    "![gradient_descent.png](images/gradient_descent.png)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "a772b2d0",
   "metadata": {},
   "source": [
    "这种方法的优点是，**在求解过程中只需求解损失函数的一阶导数，计算的代价比较小**\n",
    "不过在出发点选择不同的时候，往往我们得到的也是局部最优解$\\theta_0$,而非全局最优解$\\theta_1$,当然如果这个函数画出来是碗状的，我们无论从哪里出发，最后得到的结果都是一样的。\n",
    "\n",
    "* **迭代的更新策略**\n",
    "\n",
    "如果是最简单的线性模型进行迭代，损失函数并不会改变\n",
    "$$\n",
    "L = \\sum_{i=1}^{N} (y_i - a x_i - b)^2\n",
    "$$\n",
    "\n",
    "更新的策略可以表示为：\n",
    "$$\n",
    "\\theta^1 = \\theta^0 - \\alpha \\triangledown L(\\theta)\n",
    "$$\n",
    "\n",
    "$\\theta$就是这个模型的参数，$\\alpha$表示迭代的步长，首先我们确定了这一个点上的梯度最小值，**取负值就表示沿着下降的方向（如果求max就是沿着正值，表示沿着他上升的方向）**，走完一个步长$\\alpha$我们就完成了一次对$\\theta$的迭代\n",
    "\n",
    "如果我们对$a b$分别使用迭代公式：\n",
    "\n",
    "$$\n",
    "a^1 = a^0 + 2 \\alpha [ y - (ax+b)]*x \\\\\n",
    "b^1 = b^0 + 2 \\alpha [ y - (ax+b)] \n",
    "$$\n",
    "\n",
    "也即\n",
    "$$\n",
    "a^1=a^0+2\\alpha\\sum_{i=1}^Nx_i(y_i-ax_i-b)\\\\\n",
    "b^1=b^0+2\\alpha\\sum_{i=1}^N(y_i-ax_i-b)\n",
    "$$\n",
    "\n",
    "* **关于步长$\\alpha$的选择**\n",
    "\n",
    "$\\alpha$在梯度下降算法中被称作为**学习率或者步长**，意味着我们可以通过$\\alpha$来控制每一步走的距离，以保证不要步子跨的太大，错过了最低点。同时也要保证不要走的太慢，导致太阳下山了，还没有走到山下。所以$\\alpha$的选择在梯度下降法中往往是很重要的。\n",
    "\n",
    "在后面我们会具体介绍如何自适应设置$\\alpha$的值\n",
    "\n",
    "![gd_stepsize.png](images/gd_stepsize.png)\n",
    "\n",
    "* **梯度下降法的风险**\n",
    "\n",
    "由于梯度下降法是依赖于迭代过程的，所以如果在这个过程中有一个异常值的时候，很有可能会把迭代的结果引向错误的方向。所以实际操作的时候常常会使用[批量梯度下降法](https://blog.csdn.net/u010402786/article/details/51188876)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9972eb8e",
   "metadata": {},
   "source": [
    "### 4. 程序示例及迭代过程的可视化"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "bd8404ba",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD4CAYAAAAXUaZHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAABP70lEQVR4nO2deXwV1fn/3yfhAgkqAUGFsGotKIsEIy64gYpVK0atIj+r1K+KX5cqVKm4giJCxQVbq/2i1WoFDYJGFJWCYlEqIBgWEXAjCIECAkEhAW6S8/tj7oS5N7PfuVty3q9XXknmzsw9Z+6dZ57znM95HiGlRKFQKBQNi6xUN0ChUCgUwaOMu0KhUDRAlHFXKBSKBogy7gqFQtEAUcZdoVAoGiBNUt0AgDZt2sguXbqkuhkKhUKRUSxbtuxHKWVbs9fSwrh36dKFpUuXproZCoVCkVEIITZYvabCMgqFQtEAUcZdoVAoGiDKuCsUCkUDxDHmLoRoDiwAmkX2nyGlHCOEGAvcCGyP7HqvlPK9yDH3ANcDNcDtUso5XhsWDofZtGkT+/bt83qoImCaN29Ohw4dCIVCqW6KQqFwiZsJ1f3AQCnlHiFECPhUCPF+5LWnpJSPG3cWQhwPXAX0ANoD84QQv5RS1nhp2KZNmzj00EPp0qULQggvhyoCRErJjh072LRpE127dk11cxQKhUscjbvUMovtifwbivzYZRu7BHhdSrkfWC+E+BboB3zmpWH79u1Thj0NEEJw+OGHs337duedFYpGTklpOZPmrGNzRRXt83IYdX43igryU9IWVzF3IUS2EGI5sA2YK6VcHHnpNiHESiHEi0KIVpFt+cBGw+GbIttizzlcCLFUCLHUynAow54eqM9BoXCmpLSce95cRXlFFRIor6jinjdXUVJanpL2uDLuUsoaKWUfoAPQTwjRE3gOOAboA2wBnojsbmYJ6nn6UsopUspCKWVh27amGnyFQqHIGCbNWUdVODr6XBWuYdKcdSlpjye1jJSyAvgY+JWUcmvE6NcCz6OFXkDz1DsaDusAbI6/qZlNly5d+PHHH+PeR6FojJSUltN/4kd0HT2b/hM/Spk3bMfmiipP2xONo3EXQrQVQuRF/s4BzgXWCiHaGXa7FPgy8vcs4CohRDMhRFfgWGBJoK1WKBSNhnQLd1jRPi/H0/ZE48ZzbwfMF0KsBD5Hi7m/CzwmhFgV2T4AGAkgpVwNTAe+Aj4AbvWqlEkXysrK6N69OzfccAM9e/bk6quvZt68efTv359jjz2WJUuWsHPnToqKiujduzennHIKK1euBGDHjh0MGjSIgoICbrrpJowVr1599VX69etHnz59uOmmm6ipycjLo1AkhXQLd1gx6vxu5ISyo7blhLIZdX63lLTHjVpmJVBgsv0am2PGA+Pja5qBESNg+fLATgdAnz4webLjbt9++y1vvPEGU6ZM4aSTTmLatGl8+umnzJo1i0cffZSOHTtSUFBASUkJH330Eddeey3Lly/noYce4vTTT+fBBx9k9uzZTJkyBYA1a9ZQXFzMwoULCYVC3HLLLUydOpVrr7022P4pFGlEPCqSdAt3WKH3J13UMmmROCyd6dq1K7169QKgR48enHPOOQgh6NWrF2VlZWzYsIGZM2cCMHDgQHbs2MHu3btZsGABb775JgAXXXQRrVppYqIPP/yQZcuWcdJJJwFQVVXFEUcckYKeKRTJQQ+r6N63HlYBXBm+9nk5lJsY8lSFO+woKshPmTGPJTOMuwsPO1E0a9as7u+srKy6/7OysqiurqZJk/qXUJcOmkkIpZQMGzaMCRMmJKjFCkV6YRdWcWMIR53fLerhAKkNd2QKKrdMnJx55plMnToVgI8//pg2bdpw2GGHRW1///332bVrFwDnnHMOM2bMYNu2bQDs3LmTDRsss3YqFI6ku5Ik3rBKUUE+Ey7rRX5eDgLIz8thwmW90sZDTlcyw3NPY8aOHct1111H7969yc3N5eWXXwZgzJgxDB06lL59+3LWWWfRqVMnAI4//ngeeeQRBg0aRG1tLaFQiL/+9a907tw5ld1QZCjxhjySQRBhlXQKd2QKwqjiSBWFhYUytljHmjVrOO6441LUIkUs6vNIT/pP/MjUcObn5bBw9MAUtKg+sQ8g0MIqyvuOHyHEMillodlrynNXKDKYTFCSpJuKpLGgjLtCkcFkipJEhVWSj5pQVSgymHRbOKNIH5TnrlBkMCrkobBCGXeFIsNJh5BHOuUxV2go465QKOIiE+SYjREVcw+ACy+8kIqKCtt9HnzwQebNm+fr/B9//DG//vWvHfc7++yziZWUxjJ58mQqKyt9tUNRn3RfQJQMMiWxV2NDee5xIKVESsl7773nuO/DDz+chBY5M3nyZH7729+Sm5ub6qZkPMpj1cgEOWZjpMF47onwoJ588kl69uxJz549mRzJb1NWVsZxxx3HLbfcQt++fdm4cWNUkY1x48bRvXt3zjvvPIYOHcrjj2v1w3/3u98xY8YMQCvKMWbMGPr27UuvXr1Yu3YtAEuWLOG0006joKCA0047jXXr7D2fqqoqrrrqKnr37s2QIUOoqjp4M918880UFhbSo0cPxowZA8Cf//xnNm/ezIABAxgwYIDlfgp3KI9VI93ymCs0GoTnnggPatmyZbz00kssXrwYKSUnn3wyZ511Fq1atWLdunW89NJLPPvss1HHLF26lJkzZ1JaWkp1dTV9+/blxBNPND1/mzZt+OKLL3j22Wd5/PHHeeGFF+jevTsLFiygSZMmzJs3j3vvvbcu46QZzz33HLm5uaxcuZKVK1fSt2/futfGjx9P69atqamp4ZxzzmHlypXcfvvtPPnkk8yfP582bdpY7te7d29f16yxoTxWDZXYKz1pEMY93qxzZnz66adceumltGjRAoDLLruMTz75hMGDB9O5c2dOOeUU02MuueQScnI0j+Xiiy+2PP9ll10GwIknnliXGnj37t0MGzaMb775BiEE4XDYto0LFizg9ttvB6B3795RRnn69OlMmTKF6upqtmzZwldffWVqtN3uFzTprK5w2zYvC4jSsb9BtUnJMdOTBmHcE+FB2eXc0Q2+l2Ni0VMHZ2dnU11dDcADDzzAgAEDeOuttygrK+Pss892PI9ZWuH169fz+OOP8/nnn9OqVSt+97vfsW/fPt/7BU06x6q9tM2tx5qO/Q26Tekgx1RE0yBi7omI+Z155pmUlJRQWVnJ3r17eeuttzjjjDNsjzn99NN555132LdvH3v27GH27Nme3nP37t3k52s3yD/+8Q9XbdTTCn/55Zd1Jf5++uknWrRoQcuWLdm6dSvvv/9+3TGHHnooP//8s+N+iSSdY9Ve2uY2FW069jcd2+QWpVByR4Pw3BMR8+vbty+/+93v6NevHwA33HADBQUFlJWVWR5z0kknMXjwYE444QQ6d+5MYWEhLVu2dP2ef/zjHxk2bBhPPvkkAwc6Z/S7+eab69IN9+nTp66tJ5xwAgUFBfTo0YOjjz6a/v371x0zfPhwLrjgAtq1a8f8+fMt90sk6Ryr9to2Nx5rOvVXD8WYhZMgPT4DO9JxFBQXW7fC3r1w9NGBn7rBpPxNl5jmnj17OOSQQ6isrOTMM89kypQpUROdmUqQKX/TOU1tItqWLv01S72b6jZ5JV2uZdxs2QKTJsHf/gbnnQdvv+3rNI0i5W+6xPyGDx/OV199xb59+xg2bFiDMOxBk2x1hZcHfyLaNur8boyasYJwzUFHKpQtkq4mMQvFGMkEhUs6jYJ8UV4Of/oTTJkC1dXw29/Cffcl5K0cjbsQojmwAGgW2X+GlHKMEKI1UAx0AcqAK6WUuyLH3ANcD9QAt0sp5ySk9WnItGnTUt2EtCeZ6gqvw/iEtS12gJyCAbOdAczPEIVLpqQ4rsfGjTBxIrzwAtTWwrBhcM89cMwxCXtLN577fmCglHKPECIEfCqEeB+4DPhQSjlRCDEaGA3cLYQ4HrgK6AG0B+YJIX4ppbR2GSyQUpqqQRTJJRGhu2SNtPzIZINu26Q56wjXRl/DcK2MS6rrByvDmEkhjYzT1G/YABMmwIsvav9fdx2MHg1duyb8rR3VMlJjT+TfUORHApcAL0e2vwwURf6+BHhdSrlfSrke+Bbo57VhzZs3Z8eOHQkxLAr3SCnZsWMHzZs3T1kb4lFHpMMwPh3aAA0j93vGFMv+/nu48Ub4xS/gpZfghhvgm2/g//4vKYYdXMbchRDZwDLgF8BfpZSLhRBHSim3AEgptwghjojsng8sMhy+KbLNEx06dGDTpk1s377d66GKgGnevDkdOnRIyXvHq46w8lazhKDr6NlJmXxPl1BCQ1lslC7za6Z8+y08+ii88go0aQL/+79w992QgvvHlXGPhFT6CCHygLeEED1tdjeLo9Rzv4UQw4HhAJ06dap3QCgUomuSnnCK9CXe1cdmw3iAmsiIMBlSuqBCCUEowtLaMGYyX38N48fD1KkQCsFtt8Ef/wjt26esSZ7UMlLKCiHEx8CvgK1CiHYRr70dsC2y2yago+GwDsBmk3NNAaaAJoX00XZFkvBrVIIwRvGGNGK91Swh6gy7TrypKqww9j8vN0SzJlnsrgr7uhYNTt/dUFizBh55BF5/HZo1gzvugFGj4KijUt0y55i7EKJtxGNHCJEDnAusBWYBwyK7DQN0oeYs4CohRDMhRFfgWGBJwO1WJAndqJRXVCE5aFSc4t5+j4sliNXHRQX5LBw9kPUTL6LWYg4n6Ph3bP93VYbZX13LU0P6sHD0QM8GOZNXlDZIvvwSrroKevTQNOp33QVlZfDEE2lh2MFd+oF2wHwhxErgc2CulPJdYCJwnhDiG+C8yP9IKVcD04GvgA+AW/0oZRTpgV+jEpQxCnoSMFnpaYM2xukyKdvoWbkSrrgCevWC2bM15UtZmaZdP+IIx8OTiWNYRkq5Eigw2b4DOMfimPHA+Lhbp0g5fo1KUMYo6EnAZEnpgjbGebkhdlXWzxKa9vruhkJpKYwbB2+9BYcdBvffDyNGwOGHp7plljSYFaqKxOBX6RGkQiTIScBkKUaC7H9JaTl79lXX256KVa6NjqVL4eGH4Z13oGVLGDNGi6u3apXqljmijLvCFr+ebjovNkmGYiTI/pstggJo0bSJmkxNFIsXa0b9vfc0Qz5uHPz+95qBzxCUcVfY4sbTtVPFpEpTnepEckH23yqUs7vKvpiLwgf/+Y9m1OfM0UIujz4Kt96qhWIyDGXcGzFuDaCdp+sk0Uu0QTXrA5AWskE3/XfzGaTLIqhUkLSH9CefaEZ93jxo21abIL35Zjj00ODfK0k0iGIdCu8EJVVMpUTPqg9jZ61Oepv8pEhw+xk0hLQBfgjqO2rLxx/DgAFw5pmaEubxx2H9em0BUgYbdlDGvdESlFFOpUTPqg8VFuGKRLXJrxFy+xkkKp9Kulc0SpjjICV8+CGcdZZm2Neuhaee0oz6nXeCRRnNTEOFZRopQRnleEMGVmEVN0NxP21NBH5TJLj9DBIRmsiEFa+BOw5Swty58NBDWmw9Px/+8he4/nrIaXghLmXcGylBxXHjUYWYGZhRM1aApE4dYmd0rPrQKjfEvnBt0pQ6XoyQ0VCbpUKA6M/Ayggv3bCT2Su31Gnf83JCjB3cw7Vh9vJAStXkdGBzDVLC++9rMfXFi6FjR3j2WS39bgqznSYaFZZppAQVx40nZGBmYMI1sp7sz2woXlJaTuWB+trvnFA2Yy7ukdS0sG5XvcaGb8wMe+xnYGWEX130Q9SipoqqMKPeWOE6tOL2gXR/ySpGFi9PbNzbgri/o1Jq+vR+/eCii+C//9VS7n7zjTZZ2oANOyjPvdESpFTPryrGy/DauK9VLdBY7zVRxjzWkx3QvS0zl5XbjhRKSsu5c/oKU4OeLQS1Upp+Bl6ukZcCIG684pLScqYu+qFeStdEJVqLxfd3tLYWZs3SPPXSUi1/+gsvwDXXQNOmCW1zOqGMeyMm1elfrQyM1b46VrVAWzTztqjHT7jBLEwyc1k5l5+Yz/y1203PpR9jZtgBaqVk/cSLTF/zco3A/cPATTht0px1ltUAkzEvAB6/o7W1WnqAhx/WlC96oYyrr9bS8DYylHFXpAwzAxPKFlExdzhodHQDYmXsvHi5ficUrcIk89dutyxV51SYumWOteExu0YC6xKsbuPRbrxiu+vpZl7A+D4JpaYGZs7UVpF++SV06wb//KeWtbFJ4zVxjbfnipRjZWCstpmFYox4mWhLtMLF7WsAew9UU1Ja7rpg94DubSn+fCPhmmgTH8rylmvGySu2GjUIcDUvkPDQTU0NFBdr+dTXrIHjjoNp0+DKKyE72/n4Bo4y7oqUYmVgYrf1n/iRrWEPZQsGdG9Ln4f+Vadzb5UbYszF5goSvzI7PwoOp9BKuMY+Vm52jQo7t+ahd1b7Vsu4wWrUcPUpnVx5+Alb61BdDa+9phn1r7+Gnj01I3/55cqoG1DGXZEROBmKmlrJtEU/UGvYtqsyrEkrcS+jdPL+/Ug/rUr9GfGTCjkIrbtVWEZ/rSpcQ3ZEspmf6vQI4bBWxm78eK1Wae/eMGMGXHopZCnhXyzqiijiJhkrHZ0MRa0kyrDr6F5xLH5ldn6kn8ZjrLCLuycCu1W1xtdAk2zq18asnwlPj3DgAPz979C9u6ZNP/RQbeK0tFTz1pVhN0VIixn8ZFJYWCiXLl2a6mYofGAmS8wJZQeuK7eSP7pBgKkaJRWLcwoe/pdp0Y1WuSFKHxyU0PcGHCel9QeQ2ev5eTmWk8YJuZYHDmhqlwkTYMMGKCzU8qlfdBEIEd+5GwhCiGVSykKz11RYRhEXTpNpJaXljJ212lUc3A7jxKIXaSBYe/2JDm2YUWFi2K22B20w3Twg/U4MW11LX33Yv1/z1CdOhI0b4eST4bnn4Fe/qjPqqU7pnAko467wjPHGstNBl5SWM+qNFVGyRrs4uBO6ASkpLWdE8XJXxySyWpEfCaDb+HQ88kIrw+ckyTS2I4gYuuc+7NsHzz+vpdstL4fTTtMWH513XpSnnnLpZYagglUKT8TGaq1on5djWUHIKg7ulqKCfPIsYtTG0Xqr3BCTfnNCwm54P1kL3can/WZEtIulO03a6u0IKobuug+VlTB5Mhx9NNx+u/Z73jz49FMYNKheCCaVaaYzCeW5KzzhxvvTDcFIG+86Xpnc2ME9khLrt8OPBNDtknq/8kI7w2cnyTRTwsQb9nDsw9698Le/waRJsHWrln532jQ4++z4zqsAlHEPnETGAuM5d1DtsruBBNQLA1gZk3hlcqku4wf+JYBuYv1W584Sgq6jZ/t6KDw1pI/rB2IQ8xFWfTgmBy308vjj8OOPcO658MYbcMYZcZ23MVSm8oJjWEYI0VEIMV8IsUYIsVoIcUdk+1ghRLkQYnnk50LDMfcIIb4VQqwTQpyfyA6kE4msHBPPuYNsl9UNlJ+Xw/qJF7Fw9MA6ozDq/G6EsuqrGoKKgxcV5LNw9MB675ssEikBNDs3aLJEu8/QLkNloop+WBHbh0P2V3LH4jeYPfkaGD0aTjwRFi7Ucqy7NOxm54XGUZnKK45SSCFEO6CdlPILIcShwDKgCLgS2COlfDxm/+OB14B+QHtgHvBLKaXlWL6hSCH7T/zIs4QsGecOsl1epY9WahlIrdcdFMkaqVnlfo/9DJMlTXVLSWk5z5Us4/yPpnPDslkcVvWzJmV84AFNBRPHeRvC9yde4pJCSim3AFsif/8shFgD2F3FS4DXpZT7gfVCiG/RDP1nnlueYSQyFhjPuYNsV6wkMVuIqMksN8P7INQOiZAJ+jlfIjNrGs/ddfRs031iP8N0CFfVsWsXRW8/T9HTk2H3bhg8WDPqhaa2yBOpzmiaCXiKuQshugAFwGKgP3CbEOJaYClwp5RyF5rhX2Q4bBMmDwMhxHBgOECnTp38tD3tSGQsMJ5zB90u/abyaqDtFtB4STRl9nAYUbycsbNW+8qvYna+kcXLGVG83HLJfRB4eaB4+QxTbvh27NDUL3/+M/z0k5Ye4IEHoKAgdW1qhLiWQgohDgFmAiOklD8BzwHHAH3QPPsn9F1NDq83npRSTpFSFkopC9u2beu13WlJsmOwbs+diHZ5laPFLmk3w+1IwkqxU1EVtp1LsEqTYHY+/QubqMpDXudBMiLO/OOPcM890KWLltRr0CBYsQLefFMZ9hTgynMXQoTQDPtUKeWbAFLKrYbXnwfejfy7CehoOLwDsDmQ1qY5iRwSx3PuRLTLa6jHywIaM9wsnAL7OqBWIw2nh0oi0tdaPRzHzlrtOvVvssMtliONbds05cuzz2qa9SFD4L77tGyNCksSPW/gaNyFEAL4O7BGSvmkYXu7SDwe4FLgy8jfs4BpQogn0SZUjwWWBNbiNCdZMdigjvX7BfMa6nG7gMYMr3llzN7Lr/7b7pzxYPV+FVVh29zuqQq3mD0cn3jl3/R88t/8YuY/tZQBQ4dqRv2441LSxkwiGats3YRl+gPXAANjZI+PCSFWCSFWAgOAkQBSytXAdOAr4APgVjuljCJ1xCOR9BomsPPKnSR5brx+p/eyG2lYyQ6dzhkP2TaJr9JxpaXxMzji5x08OG8Kc5+5ji7TXoArroCvvoJXX1WG3SXJWGXrRi3zKeZx9PdsjhkPjI+jXYokEE8FHa9hAqs86G4kel68ZqsHjN1II1YBFFvGLhGxbat6qpCeKy03V1Rx1E8/cvPiN7hqxb/Irq3hzZ4DefbUK/n4/270fL7GLmVMxipbtUK1ERPvF8xLmCCemLFduTcZ8//lJ5q3yerhMqB7W/pP/KiuTZOH9PHdTi/k24SC0m6l5Q8/8OTH/8eFS98nS0pm9DyHZ0+9ko15R9nmqLdCJf5KzipbZdwbMclexu3mYWCUS+oVgPJyQoSyRVTNULMi0RKYv3a75XtD/VqkM5eV1zMyEy7rFfeiMydGnd+NUTNWxF0HNaGUlcGjj8I//sElEqafMIi/9PsN5S2PAPyPaFJWczWN8FPRyyvKuDdiRp3frV5K3lQal1iPTg9dVFSFCWUJWuWGqKgM206AlldUWeZeiX24mNVldWNkgggp6PsHWQc1sFDHd99pRv2VV7QqR8OHk3X33TT/MQvmrEMkKD9RIsJR6Rr+SYb6SRn3xk7sbEoKC9zYTZyGayW5TZvUVSuySqkARE0Og/VQ34+RCTKkEKT6JZB2ffONVp/01VchFIJbboE//hHyteOLOgYTNknWiDHdwz+JVj+pfO6NmElz1tULC/jNtW5XR9VtjVUnz834uhuFi5P6wC7JlhVuVA7JqCnrp12WrF0L11yj1SidPl3Lqf799/D003WGPUiStSCrsed9V557Iyao4bGdhwTu0xQ46c2NRjd2WGtXEcoKP3FPp2uWKm/R12f51VcwbhwUF0NODvzhD3DXXXDkkQlqpUayFmQ19rzvyrg3YoIaHjt5SG7j2mbGVsfM6BqHtVZhGru++DEyTtcsVZOFnj7LVas0oz5jBrRoAXffrRn2JKYBScaCrMae912FZRoxQQ2P7TwkL96TMd84HFzooy9yAizDHVZ90aWOViESrznhna5ZqrxFV5/l8uVw+eXQuzd88AHce6+miJkwIamGPVlkRD6eBKI890ZMUMNjJw/Ji/dklybBLtzhRepo7LtXnK5ZqrxF23YtW6Z56m+/DS1bwoMPwh13QOvWCW1TqkmHfDypxLFYRzJoKMU6Git2BSIA3ytTjfgpOGKnqAk6la9Rn2+2wjUlxTKWLIGHH4bZsyEvD0aO1CZL8/KS2w5FwoirWIeicRCvHrh5KKvOgJvpteM5d0lpuaWRtgt32L1m58V7vRaxDzfJwUVWQTxEzBZ22Z73s880o/7BB5p3Pn483HYbHHaY7zYoMg9l3BVxKTzMvPb91bVR+8Qzeaaf3wq7cIeT+sZsotPPtbDKBx9EeUWrhV2m7fr0U82oz50LbdrAxImaVv3QQ+NqgyIzUROqCluFh5Nm2+rYEcXLA9F42y1scpocc6OFj/Xu/Wijg5hE9VJIpF67/v1vOOccrcj0ihUwaRKsX6+pYJRhb7Qoz11haYR079DOi/Ub+oi3bYBjHDs226MZsZ6/H0PtdxLVKk7vqpCIlJz6w0pGTHsN7vkSjjoKnnwSbroJcnNt31fROFCeuwdSsfLQDfG2y8oI6cWvjcR6sU4GLN4VgVbnzzek6rVDlzpOHtLHlSzOz6pVP5K72LKDsbIGYyGRKKTk9PWlTJ92N6+9fh9H796irST9/nttwlQZdkUEZdxdEk9hi3Rv14Du5hpnq5zjXtMAxKPxtjp/5YFqT300augF1gVC/Bhqt+c24qYASVQhESk56/tlvPnqXbw6/QE67N7GuF/dwmdzFmsKmJzGsTBH4R4VlnFJuqYpDaJdVmlydWVGLFZpAILIT26mVJlwWS/GzlpNRVW4br9dlWHPIR83E7t+tdFeJ43dPPDa5+VQ1Kc9Ry6YR8vHJ3D8pnWUH9aWe8+/lf+c/mtGXNSLwY1Es63wjjLuLknXPBV27XIr6bM6R42U5ISyHXOv6IbNSu/udkWglVJlwmW9aNGsSZRxh/gernbXJpVL43VymmTxRLMyKCzk1C++gC5d4Pnnyb/2Wh5t2jShbVM0DJRxd0k65KkwM0hW7WqZE4o7YZeupTZqrI0x9NjzxLsi0G4UEuTDNR1SwZrl0REAspah5V9w9+fTafn1V3DMMfDii/Db32ppeBUKl6iYu0tSnafCKrY+oHtb03YJYZ2wKxa7vhUV5Ne9HquxNot5e83VYsTOgPuZ6LQiHVLBxsbpOxzWjOLWm1j//n08OnUsLWUYXn5ZS8d73XXKsCs8o4y7S/xMmgWJlUGav3a7absqKsOm53FK2GXWt2QZQzsDHuTDNV1CbEUF+SwcdRbr+/zMp1PvoN/d/wvhMEydCmvWwLXXQhM1uFb4Q31zPJCMWKwVdgbJrF1WE5xeE3aBeeIvszbFm8LALr96EEmg9PZZZVNyOwoIpHRbdTW8/jo88gisWwc9emj//+Y3kG2vPlIo3ODouQshOgoh5gsh1gghVgsh7ohsby2EmCuE+Cbyu5XhmHuEEN8KIdYJIc5PZAcaAm506laGJ0sI0/3NPN1QtmDv/mpPeviS0nLLynvGNnmRZFr1124EEa9BjdWVx+J2FBC39LS6Wgu3HH+8Vv2oaVN44w1YuRKGDEm5YU/XtRwK77jx3KuBO6WUXwghDgWWCSHmAr8DPpRSThRCjAZGA3cLIY4HrgJ6AO2BeUKIX0op7UW9jRS3k3tWhSxqpDTdP9bTzW2azd4DNXWKE7eTiHaerlEf71aS6SZ1r1kir3gnQJ105c1D5n5O7EOl8kC1P+lpOAz//KeWxOv776FPH3jzTbjkEq0IdRqQDhPNiuBw/FZJKbdIKb+I/P0zsAbIBy4BXo7s9jJQFPn7EuB1KeV+KeV64FugX8DtbjC4jWfrXq1ewMJpf/2YhaMH8tSQPlQeqG/Y3MTN7eLQry76oc67s/KIY7f7id/Hk/vGTT/goG4+tvZrrJe+y8NcBgAHDsDzz8MvfwnXXw+tWml51b/4Ai69NG0MO6THRLMiODzF3IUQXYACYDFwpJRyC2gPACHEEZHd8oFFhsM2RbbFnms4MBygU6dOnhveUPBaqWhk8XJP5wF779vJ6DnpsXXvTggwW9CaHQkbxVPr1E/uG4iOz+flhiwNs06sB+5mFalOvbDZ/v2ahHHCBNi4Efr1g7/+FS64AEwe0OlAukw0K4LBtdsghDgEmAmMkFL+ZLerybZ697SUcoqUslBKWdi2AZb4cotXiZ8fSaBT0is7Rp3fzTLmrlMVrjE17HAwbFRuY9id2uE1983YWavredx79lUTynY2qsZr5daoRcXr9+2DZ57R9Om33AIdOmh51RctggsvTFvDDv6+W4r0xZVxF0KE0Az7VCnlm5HNW4UQ7SKvtwO2RbZvAjoaDu8AbA6muQ0PrxI/P5JAq5tTRM5nR1FBPlef0snRwFthZoBj8ZO616i7j6WiKlzvPcO1khZNm9RN1pqFtyD6WrkxanWTvt1bawm8jj4afv976NqVhc+9Rv+Lx9F1fjX9/zQ/7ScnU72WQxEsbtQyAvg7sEZK+aThpVnAsMjfw4C3DduvEkI0E0J0BY4FlgTX5IaFV/187P55OSGah7IYaZM/3eymFcDVp3RyNVH2SFEvrj6lk6VBJNIOLwZYJ1sILj/RWWLarMnBr2qr3FBUIW237K4K1y2weuLKExwNmVNSNAEs/P3JFM0vhq5dYcQI6NYNPvqIkqdf44byPMp370urRHN2pHothyJY3MTc+wPXAKuEEMsj2+4FJgLThRDXAz8AVwBIKVcLIaYDX6EpbW5VSplozGR9Xir2WOVysVI3xKsRLyktZ+aycktDnRPKZuzgHqbvYZdQDLSwzcxl5RR2bu2qMDbAvrBW6clKF988lGUaX7dKeGZ1TfS/75y+ol7fcw9UcevaudD1Oti+XSuWMX06nHmmdt6JH6VlojknUrmWQxEsqkB2krErJu31pvJTNNoP8RSaNuuv1XnM2uzUR7MHJQRTlNusDy32V3Jt6Wxu/PwtWlf+BIMGwYMPQv/+Ucd0HT3bdI5BAOsnXuS5DQqFGapAdhrhJUWv08IdP0Wj/WB1PgGWDxFj21tGQkcVlWHPahmn7XaeZtyrSCMUFeTT5Oef2PzIJK74ZAat9v3Mf/sPgMcfhVNOMT3GSp2Tl3swR0wgK10VCguUcU8ybo2YU8hFXzlqZiyDVjd4zYgZ2/aKqjA5oWyeGtLHc1oEv9k4vYYXLA1tRQU8/TS/njxZ+/vii+GBBzjqpJNsz2c1INa3qwVDikSTPisoGhhWi2vcys2cFpRYadfdKGC8YqWiGNC9reuiznrbrSZ3yyuqTCeE/Zaw87KE3myx0sSp/2Ht8BHQuTOMHQtnnw3LlsGsWeBg2EGbvLXbrhYMKRKN8twTgJVXtnTDTvbur663v5mxcvLwrV6XBO/5FRXks3TDTl5bvJEaKckWgr6dWjJzWbmp5+mU5AwOJjazKgxtLJyh7+8mfOHHIzYa2laVu7l+6dsMW/YOhx6oYn7PM5hUeAW7u/VglDiybhm2E04jjmQsGFJhn8aNMu4JwMorm7roh3redqvcEGMu7uG6gIZuHOwKbECwN3asWqZGShZ+t7PefsaiznZt10MmZpOlZvMPXkIsfsoObq6oonXlboYveZNrvphNTng/73U/nT+fNoSv23bRdvIYNrHLcAmJL/6iwj4KZdwTgJ1XHUtu0yamWQ8HdG9L8ecbCdccPCqULeqMg53xCPrG9rIMf3NFFU8N6eOq3F683qt+zfQqUXaaestzbt3K+P+8TNGiWTSrDvPOcWfwzKlD+LZN/ZQYXqSMdiOOktJy1yM4v6RrzV9F8lDGPQE45WMxotc6jTXGxUs21jdWhn/tjEf/gDXWXkIF7fNyXIdS4vFeY6+Z02KpeufcsgUeewz+9jeGHjjArB4DePrkK/j+8A625/FyLdxkuNSxGsH5ReWJUSjjngCs6mNaKVvMvKxwbf29w7UyykBbhSuCvrHdPqyMk7luQilWaYwrD1RTUlrumIrY7WgiyiPetEkz6lOmaLnVr7kGce+9yD257J+zDmFI7Wu1EKqktJyH3lld93peToixg90ZZqt26yO4oEiHmr+K1KKMewIw81wHdG8bNQEJB42OVaZHM5wMdElpOVkWIQq/N7aVEY7F62Suvu/YWavr8szDwfS7xn10jKEYJwQHS/QVHV7D91cOo8ObryFkLR/0HUSzB+9n0MWnae8T815Wi80GdG/LqBkrosJlFVVhRr2xwra9+vcgWWsTnGL+ioaPMu4JwsxzLezc2jRU4dZYgb2B1g2SmWF3c2NbTcLq/TB6q2Z4zfUC1PW/IkY66KbQhx11K17LymDCw9S++BIdaiUzep3Ls6dewaaWR5Kz5CcmdDAfIViFlibNWRdl2HViR1Vm7Y1VBxkJ2qMOoiyhIrNR6Qd8ELTEzMxohbIECKIMidMSequl+tlCMPTkjsxfu92yzW7TIpSUltfztN20zQ63S/Xt0iDEtuXpfocx6O0X4eWXqRFZvNbzXJ495TdsPuyIqH29pmqwaquX9sYa+HiunaJxo9IPBEgiJGZWXpbZNqvkWnbev56cy67NXtQVLZo1oaIqXKdQccov44Tb+LBd6EJvy8nVO3jsq9l0njATmjThu99cw/VHDKAsp7XpcV6LfNuFVty2V6I9VJRHrUgkyrh7JFESM6sJSKdzuglVWBW1MLbZTZk8M4WKHu6Jp+9u48N22v6Fl+Zr9UmnTtWKTv/+9zBqFNe+stbW2zcr8m22+Ewf9bTMCZGdJaiJmfAOZQlv7Q0wsZtCYYYy7h5JN4mZk2okJ5Rt+bqxzVY6cWMOd6sH20PvrI4rTOU2Pmz2EOhZsYnnVn4A982C5s1h5Ei46y446qhIH0st3zf2AeJm8VlFVZhQlqB5pOA41FfLGEdSZiEYNampSAbKuHsk3SRmdg+V/Dz7nOotcw5mKLTSiRu3W73Xrspw3USr3zCVG+mk8SFwyDdr+OPnMxj45b8RubmaQb/zTjgiOqZu9XllC1Evzu128Vm4VnJEblNWP1zf+471/o3Hxhu+Uii8oBKHecQpkZXXpFXxYvVQ0Yf+RQX5jDq/mzZBG8PeiJ5c39/qPE7vFUsiE2AVZf3IwqXPMufF2zinbBninns0Rcyf/lTPsIP15/XElSeYLqpyi9WDwGokpa8BUIZdkSyUcfeIXSkys+yCiS6tZma8QtmCvfur6x4wAIc0rz9IC9fIOiNs99DSH1h6mMENgYepvvgCLr0U+vSBuXPhgQc0oz5+PLRpY3mYl9JxVhkrzbB6ENh5/yrjoyKZqLCMBXaqCasQQiryecTGq/NyQ+zZV10nVdQfME5xdzvFjlWYwY7AwlSffw7jxsE770BenpZ+9/bboVUr16dwm3jM6+IzM5K5UEmhsEMZdxP8yh3dKE4SgdF49Z/4Ub2FRlXhGssJUxk5JnbBko5ZnhonApk0XLwYHnoI3n9fM+TjxmkKmJYt4zuvA14Wn5mhrzhOxkIlhcIOZdxN8OuBu1GcJBo7rbuVcsbu4eXV24x70vA//9GM+r/+BYcfDhMmwC23wGGH+TtfHMSO3p4a0sfVpO/SDTvrpXdWKhlFslExdxOsDJpeLchqstSN4iSR6KX3zNBjzVYTp1aToF68TV3r7cuwL1gA556rFZouLdWSe5WVwejRKTPsfudPHinqxVND+riK8ysUiUJ57iZYxU31cnBg7u3mOxTQ8IOXVAdWpfdAy7Q4sni5rbHWH2qxxa1D2aJeGgSQVIVro443y69ii5Tw8cfw8MPa7yOPZNUfHmREy358vwPaP7MkZQqTeOdP3Mb5VbUkRaJw9NyFEC8KIbYJIb40bBsrhCgXQiyP/FxoeO0eIcS3Qoh1QojzE9XwRGKlmog1nLHerp96n3Z49R7tQii7KsN157AiLzdEn4f+xYji5XXvWVEVBqnlGzd6oftiDLubNtQhJcybB2edBQMHwrp1MHkys976lMuan8x3ldS1ddSMFQlTG9nJVpNVBi/Z6ipF48FNWOYfwK9Mtj8lpewT+XkPQAhxPHAV0CNyzLNCiGyTY9MaM/mclUdsvNm9yO7c4LWIcrwTdrsqw/USgoHmkec2bcL6iRfVaefdFvqOQkqYMwdOPx3OOw++/x7+8hft9x13MGbu9/UyLoZrJA+9szqufpnhZFh99c8jqki2IpE4hmWklAuEEF1cnu8S4HUp5X5gvRDiW6Af8Jn/JqaG2GG1VYa/2JvdS71PJ5xi/7FDebd514Noi6d84VJqqpeHH9ZUMB07wrPPwv/8DzRrVrebVTphuzTDfnEKu4w6vxuj3lgRVTTFLH9MPKRbKgtFwyKeCdXbhBArI2EbXXScD2w07LMpsq0eQojhQoilQoil27dvj6MZySHokIsbrLxEPfYf63GajRzyDCkGgmyLq1GKlDBrFpx0Elx0EWzdqlVA+vZbuPnmKMOeSMzCL64Ma+zsdMCip2SMDhSNF78Tqs8B49BCo+OAJ4D/wfzrbxrRkFJOAaaAls/dZzuSht/iB/FMmLkt12f0OGNHDl4KXFhh9RCzHKXU1sLbb2ue+vLlcPTR8Pe/wzXXQChkeU3yckKmYSGA+0tW8UhRL89tt1qz0NLivXTDalaUQ1/Ra5d22cvnrKolKRKJq2IdkbDMu1LKnnavCSHuAZBSToi8NgcYK6W0DcukS7GOZBTh8FqYwW2ZtthCEVbnaB7KqqdyscNT4ebaWpY88TytnnyMY//7PRsPz2fb7Xex6VdFPPbR91EraI3hDv2aAPVCIbF41dFbhdNa5YbYF661/GzsinLE5mIHTD/ny0/Mty2QAt6+c0pZo4jFrliHL+MuhGgnpdwS+XskcLKU8iohRA9gGlqcvT3wIXCslNLWbUwH4x6EIY7FyrDEk8/brhqRG8PnVM1IHxl4MqI1NTBjBj/dN4bDvlvHd6078JfThvDOcWeS1aRJvYpSVm1fOHogJaXljHCoKevlc7Gr8vTUkD6WxtJLFaXmoSzTeYEgKy4l4vtp9T7qAZI5xFWJSQjxGnA20EYIsQkYA5wthOiD9t0tA24CkFKuFkJMB74CqoFbnQx7upCIvDBuJ8y83FB2k6Zu0iS4SRHsur81NVBcDI88AmvWsOOIztx38Shmdz+d2ixtfiK2qIUVuiEtKsh3NO5V4RpGFC9n0px1ju21S9FsN/ntJSRmFfKyC595JRl5ixJRZUyROtyoZYaabP67zf7jgfHxNCoZJKMqvZvc715vKGPs3+zcTjd8INWBqqth2jQtI+PXX0PPnjB9OgOXNkcKf3P0Auomha3SOMTixvj4jWvHXme3bXLC7/cpGcqaVCS+UySORpl+wEzj7DW1qxsGdG9b77xuq//Eo3W2u+HjUv2Ew/DSS9C9OwwbBjk5MHMmrFgBV1xBVpb/JQ3GlLhDT+7o+jinaxXP2gNdEpkTyrY17Hk5IdepglvmhHzl+0+GskZJMxsWjTL9gJlBNbt1411dOnNZedR5BXD5idGhAK83lBv1i90N70v1c+AAvPIKPPoorF8PfftCSQkMHgxC1I2C4vVs9T7rqpjXFm90dU4n4xPP2gM3ZQzHDu5Rt69dquBQlmDvgfrpmPU22pEMZU26VRlTxEejNO52xiAvJ8TuqnDck0lWD5D5a6M1/V5vKDfGxk3Iwa5furH+8cefuP67f3Pb4hnkbtmk6dX//GdNsx7JdOnmYdMqN0Ru0yZ1hm/v/mpbGSJoBl438saapGYk0vh4maNwShVceaDaNB2zm/kDv1JcLyhpZsOiURp3uxh7i2ZNWD5mUNzv4dYj93pDBTkhajaRCzBm+jIGL3ufmxfNoP3PP7I8vzuVf/knp916dZRRtzO4xr7ESimtHgiVkbJ/se3XH0ZWipFELyTzO0cR+xDtOnq25b5uvPhYA6+Ho4Iy8Ml4gCiSR6OMudsZg6Dii04xUn3V5Mji5TQPZZGXE3IVE24eMv/IjDVT3WA27zC2eClf3zeeOc9cx7i5f6P8sCP47ZXjKLp6EqP2tK/nrTsZdrO+6A+FqnBNvbj0rsqwbeKsooJ8Lj8xvy4/frYQ9cJcQWOVRE5PAeElyZfTCMNp/kAlGlN4oVF67kUF+Tz0zmpTbXJQQ3w7jzzWA91VGSYnlO1YDOLq5z8zXYCUhf0DywxjeKd5eB9XL/+AmxbP5Ii9u1jcsScjf30nn3XqXWfQjQ89p9AQmHu2sf02i6ZXhWu4c/oKoL5Hqs9j6HH4GimZuaycws6tk1LGUJ9419vtVSroJvePnXORaDWLkkI2LBql5w4w5uIeCc0VY6fS8KOQKSktZ+F3O01fq0W78b14cJsrqsg5sI8bF7/JJ3+7gQc+eoFv2nRkyNAJDPl/E/ms8wl1hh2iH3pOoxuBphSKxc1DATSjbeaRpiqLYlFBPgtHDzTNDurl/Y3fCSvc5Nt3u90rKktlw6JReu6QnPii1cSln5t07Cz7tLeevKyff2bUihKG/Hs6h1f9xIIuBdzc/yqWduihyfqqa+vd5Hv3H4yH281ZgObZmnnUXoyQmUeaaqleEO8fz/xBotUsqb6+imBptMYdgk3P6wWvN2lJabllQi0jjkP0n36CZ56BJ57glp07WXBMIZNPHcIX+ccB0bK+2LBVRVW47uHhJrxg1harZF1WxBoVt9fNasVvvEvrgzSufpyLRKtZlBSyYdFowzKpxOtCIi/DYlOPuqJCy9DYuTPcdx+ceiosXszON0rY2qNvvbBRUUE+uU3rP/eNBtsYcrLCaJxLSsvZe6C63j5ZQJbFSWKNipvrdn/JKkYaKknpI5r7S1bFPRkZdNpnPdxjLIJiJDZVMRBoMZhYUpHWWpE4GrXnniq8em1ehsXGZfzs3AlPP6397N4Nl1wCDzwAJ56otQPrEI7TEN046nFTyMQshS5Ay0jWSTceqdN1KyktZ+qiH0zj4mYLorxORiZTKmg1uTnhsl6+k845oaSQDQtl3FOEl5CQU4zbiAQmF39GxZ0lXL7wTQ49UMXmgRew7sYR3F/WhM1v/Jf2cz+KK+FWLG7CBVYPi4rKsCuj7cbg2BUIt1rp6jWenKxQXqryvKQqVKkInkZr3INIbZqs9KhWxjP25m9duZsbPn+La7+YTe6BfbzXrT/PnDaE79odDSurCde6X/buJb5bVJDP0g0767xjXX8O1JUDzLJIvKU/LKyMihd5np2htkr8la7xZDW5qYiXRmncg9DzBq0JtntQWHm2uva6zd5d3LjkLa4pnU3z8AHePe4M/nLqEL5p21k7uUk4xMkL9DJEN9OfFy/ZSPHnG+tCMWaG1U0814sHazXaEGjJyGJzvaRzPFlNbirixVWxjkST7GIdQRTRCLIQh5dCDMaHwDHVP/H/PpnO0C/ep2lNmLePP4u/nnol3x3uLquiXfUmLzgVADGSLQS1Uroe6cRbEUkAV5/SiUeKemVUIYpkFedQZDZxFetoiAQx5A1q2FxSWs6d01e4muzTb/jDdm7lwcUzGbpiDk1qqnnvhHOYfNJv2H/0L9i7vxpcyg3b5+UEYvC89LlGSsocHijGNlmFc/QUABA92Tjhsl62I6BMMYxqclMRL43SuAcx5HVbiMPu5tSNtdvJvpeLP2H03Fe5asW/yJK1vNljIM+eegXVXY+pGy2YeXyhbAGSqNqkAuhyeE4goSUvE77ZIlr3GHuNYlPlWhl2q5WiXvLrpANm/Xequ6pQuKFRGvcgFoM4ncNNTN5pOX7dg2LDBpg4keIpLyCQvNHrXJ495Qo25R0FgDAYVrPJzSEnaWEao0xQgmk6Az+KDDeLmnRqpIzKKBmbq8VMygjR4ZxEVM1KBWbfkVcX/VD3usrtooiHRmncgxjyOp3DzUSgnTHKCWUzpmcO3Hgj/OMfIASzT7qAxwsupbzlEVH7xo4WzJJrNQ9lWcauY/EjDwQcQymg5Xa3Sx5m1cZaKevmB9zo6jMBN7l2VJk7hV8y3rj7jRkHEX+1O4ebmLyVF9q14r/8Y8tcOk+YAdnZcNNNcPfdiB+z2PnmKrDINGlXV9WNV33wnFkcc897dZ7/0JM71hXOsMJ4LezylkuJp7boGA13Qykq4fYhmmkjEkV6kNHpB9I5v7Wbmpexy7277iznidlPMnfKcI567y3+c8FVXPKHV+h6yAX0n/oNoC0/z8sJ1R3TPJTF0g07XeVXd0tluDbK83910Q/cX7LK9fFWfderXDnhVHfWLuNmoohNBRDEd8ztSCPTRiSK9CCjpZB2ckRdBx4rlUuW+sCtlK2ktJzp//wXV8x5hcFrFnAgO8SrBRcwpd9lbD+kddQ5c0LZXH5ifj29ttkEYyx5OSH2m2R7dEu2EHw34UJX+9r13al6k97HdJpUTJQs0U2JQiV/VNhhJ4V0NO5CiBeBXwPbpJQ9I9taA8VAF6AMuFJKuSvy2j3A9UANcLuUco5TA/0adzsNdOwKzlCWAEFUfpOgbhzfWQhXr4ZHHoHiYqqaNOPlggt5vt9l7GiRZ/leVist7RDAU0P6AAcfbnm5Ifbsq45S0DhhlDC6UQJZXRMzLbrEe5nAZGHlRHjV7Juh1DKKeIhX5/4P4BngFcO20cCHUsqJQojRkf/vFkIcD1wF9ADaA/OEEL+UUvpzFx2willnC1HPGzIzYkFMVjmpYkzPvXIljBsHM2bAIYfA3Xdz+t5e7Mht6fh+Xg07HPTq7UYydpOgEC1hdKMEsup7Juq3rWLe+vWKR9WSSdp7RWbhKiwjhOgCvGvw3NcBZ0sptwgh2gEfSym7Rbx2pJQTIvvNAcZKKT+zO79fz91quOwl9OBllaZx0lL3oK08adOVqsuXa6l333oLDjsMbr+d2QOH8Ojiba7j5VkCzJxtu9CMWUhGH7UAjqESgNxQFo9e1hvAdNEV+Fud65ZUri51uwI3kf1XKMxIxArVI6WUWwAiBl7X5uUDiwz7bYpsM2vUcGA4QKdOnXw2A5o1yaozWq0i6WPdGCsdt5NVsQ8S44SjGVHe3rJlmlGfNQtatoQxY+COOygpq3StD69DaouSjOGlUJagaZMs9h6of56cUDZC1FeoVIVrGDtrtes4fGW4lhHFy233CWpCN5ZU1/Z0q+MPQtWSSSkSFOlN0GoZs7ILptZPSjlFSlkopSxs27Z+vU0n9BveWNlnX6R4tFnRgVCW0FZqGvAin3Nb/1OnfV4OLFkCv/41FBbCJ59oBr6sDMaOhVatbM+ZLQQhk0+nFm3eQA+T5OWEQBBl2PVe6iqSCpNC4KBVV/I7wWqGnks+aFJd2zNWnRO7ylYnXlVLOqu/FJmHX899qxCinSEssy2yfRNgzFrVAdgcTwOtsLvh9aFxkGoZL17ZqVu/ZvKCt+Gef0Pr1jB+PNx2mxaKcXnOGinNkjlGva575bFFMGIP85IeIB4kMKJ4OZPmrAvU40yH9LfG2Lif+qdGrLzzVOVwVzRM/Br3WcAwYGLk99uG7dOEEE+iTageCyyJt5Fm2N3wbtLnesXJQGYLQd+NX/LHxcWc9O0X0KYNTJwIt9wChx7q65xOOC1O0j0/M/mkHX4UOWbvC8GETfJyQ1H1XHWC1n+7DYnEMylsF2JKh4eYouHgaNyFEK8BZwNthBCbgDFoRn26EOJ64AfgCgAp5WohxHTgK6AauDXZSpmWOaGExGft4q79N67ijs9ep9/6FexokceTg27khR7n00q2YtS3P1FUYG7cveRk8UtVuIapi3/g6pM78e6KLa4KVMdj2I3vO3bW6kBkpnv21a+9GsoWga5I9RrX96tysfPOVQ53RZA4Gncp5VCLl86x2H88MD6eRrnBagm61eRhvENbo7dWXlFFNnBy2XL+sKiYwg2r2NaiFQ8PvJFpfc5nX6g5AJUWBsLoIbbMCdE8lGXqmYK1OkbHzeIkKeG1JRsdJ1ji9dhjqagKH6zn6pNJc9aZylhbNG0SaKgiWSERO+/8qSF9GkRaBUV6kLHpB6yWoFtNHpZXVMW9dLyoIJ+Fdw+gbGCI7z6dwLTi++m8azNjzr2JM256gRdPuqTOsOvETvzFTppVVIXrJoLNqJVa38wQwNjBPbj8xHzTmWwjNbXSdsFSfl4OtR4Me6vcEJOH9LFsm05s370u4bcyhm7SGHjB6n3KI2G+oLBLS5GKtAqKhktGJw4zGxrbySCNCgT9eNdICR98oCleFi1ia8u2PHPezUzvfR77mzS1PbS8osq2lmhVuMZWL282StErDBUV5PPQO6tdZ3w0Q0Ddykg3cwA5oWzGXNyj7vqXlJZbyiR1o+lXzpjoUIU+irK7fkHOHwzo3jYqra9xu/4eypgrgiBjPXcrzGSQsXiS0UkJ774LJ58MF15IZdlGxlzwe864YQr/7HuRo2GHg1WDJNbxbF39YkQfkpt5dE8N6VNXOs4qpOMWCcxcVs6A7m0dr50A+nZqydhZq+kyejZdRs/moXdWk2um2+SgEfYrZzT7PIMKVRhHUXYEKbucv3a7p+0KhV8y2nM3I1bJYOWROSoQpIS339Y89dJS6NoVXniBC7Z0YMOe+hN8tqdysY9VsjOn5fxBGZ2qcA3z127n8hPz6wp9mGFW5GNXZZjsLEEoS0SFfoxG2K8SJJHpCrysXQhKsaIUMYpk0eCMO0QbQs+FHWprtfQA48bBihVwzDHw0ku8ffzZPPbR95TvcX8TusnWCNEeupXRspLpuTEKrSykhLGUV1RFFfrwQk2t5LDcELlNm5ga4XjCK4kKVXgxqEGFgdxeB7VSVREvDS4sE4vrYX1NDUyfzu5ux8NvfsP3G3/k4StG8/ZrH1JywnncVfKV7fA9Py+nboLRGDqxmnDMFiJq0gywnGy0W7loZ3QmD+lD2cSLKH1wkOPEJ2jKnHhkmRWVYRaOHsj6iRfVq2WayPCKX6yunVM++Xhwcx3USlVFEGR0Pne32HpBEaPOuHGwZg3fHd6Rp08bwrvdz6A2KztyI0qqbBQtdqmD3aS4BUwlcHpec6uHin78yOLlpiMEYyIrp9zhsflqrLAbjTglzko3b9RqpWmi88k7XQe7OgUqMZnCSCISh2UUpsP66mp4/XUtn/q6ddCjBw8MfYCpHQqpzTroWTl5sk45yGP18bEFoe95cxXNQ1mmk41WxaJ1NldUUVSQ76hU0TEmWWvRNJtQdha7q8K0z8th7/5qywVOxrzlA7q3pXjJxnqySjeLimLj5/p8QaoMfKrSDzuFmVRcXhEEjcK4w0FvaevOPVy3/lPuWPwGh2wsg969tbzql17Kq/e+71lSaPSMnSZDzTwyuxQCTm3Rwwr5DnFcMw+1VmoaeTd1T5+48oQoY1TYuTVjZ62uexjo2TidjGKqszuakY7SQ7VSVREEjcK4l5SW88AbpVxQOpdbF02nc8V/+eqoY/j5iRc4ecR1kKVNPXjN9dIqN1R3/lEzVtSFNcorqhg1YwUQbbSC9LyMcVqngtFWMsQRxcu5c/oKhp7c0bLveTmhesYvEUvv08HApkvYqKEUAFekloZv3A8cYN24J3j/w6l0+GkbK446locvv5EPj+lH/oFcFmYdnFP2kusllC0Yc3EPAB56Z3W9eHW4RjJy+nJGFi+vMxRBZWcUgqgYf2x4oWVOCCFgZCRDo9176gWw+x/Tmp17D9QzKGMH93DdLifjGES4IVEGOFGjCj/tzcRqVYr0o+FOqO7fDy++CBMmwMaNlLbrxtP9r+Ljows164h5FSbjzWh3ZSYP6VN3s3WxCWnoWBW39kuZRfUoN0WXzcgWgieuPMG3QXFTRNqpopHT/EWiClXbtS2eScxEtlehgMY2obpvH7zwgpZut7wcTjuNkefexlttj68z6jpmMUw3GnldVqinFHCDvkhowmW9ogxo5YFqXytMjQm5jA8kcKetj6VGyrjiz25CLk4jIydvOZFhnURMYqZ7GErRsGk4xr2yEqZMgccegy1b4Iwz4OWXYeBAzlq+mQ98xDCtYp8Durf15R3r6pbYDJFO5evM0I2g/ne8owGr6kJucWMc9X5b1WAFe+OXSBVJIiYxlepFkUoyfxHT3r3wxBNw9NEwciR06wbz58OCBXDOOSCEr2x7ujesJ/XCcNz8tdt9GdOWOaF624oK8usmZr2gG0Gv5f+sOOXoVnEdb2UEJUQtyioqyHfMPmll/OwyKsZLIhZZJbK9CoUTme25L1oEgwfD9u1w7rnaYqQzzzTd1UvIwawYtjFFwEgfnjZo6X3vL1nFI0W9oraPubiH75FAUJTtcHeuktJyUxnkqPO7MeqNFaZphWPDLU4Ty1bGL5EqkkRMYirViyKVZLZxP/54zZj/4Q9w2mmBndYqVvqH6ct9hVCMTF30A4WdW0cZDbfJzmLRjaCTAqdVboifqqptc8a4eVCUlJbXM+C7KsPc+cYKhvbraF4ePYIx3NLlcGvjbmf8Eq0iCVrzrlQvilTScNUyPtBDMX7kim6ThIGzAsNJVQIHVRdgH3PX91u6YadpHnG3bXJql1PFKNCu0VND+limS9AVO8r4KRTuaFxqGY8YDboXA20kP7Is3yldgI6Tl2w2nA9lC1o0bVKXLiDWA9S9w+ahLPZX11IrNWN5+YkHvdHXlmykxsQCuw0V2LXbybCDNtKwK4xRG1HsKMxJl0VWisygURv32Ni6H8MuIMrjdWPgnSbU/A7nJbAvXFv3/jVSMnNZOYWdWzNpzjpTw54thGvddTyLsPQHiN18hZpotCYdUzco0ptGHZZxE/5wwphUy7jcf3NFFblNs9l7IDpcEuQilvtLVrl6mOTn5VjG8p0WchkfLE6yzdhiHbHZL63y6+j7PmVYGBYvQXm56eItq0yRCjMSFpYRQpQBPwM1QLWUslAI0RooBroAZcCVUspd8bxPvMRT6MIJfZJS96QmXNYr6mZLlHG4v2SVbQzdiP7eZsZBlyoaDbidh2gbuxdaLprY0JF+DUYWLycvN2T6ENDrwQZBUF5uOnnLSjOv8EoQYZkBUsofDf+PBj6UUk4UQoyO/H93AO/jC7sb1E2YwUsc3mwBTiKyDpaUlrs27ECdobWaeDUmOnNaVflIUS8KO7c2XYgUrpG0aNaE5WMGRbXVqLDZVRkmy+IhEBRBrQxNpxWmKlOkwiuJWMR0CfBy5O+XgaIEvIdr7G5QN8W0zQy73TF+PamS0nLLSkyx+xlXpzphVmTbjHCN5A/Tl1s+7GJXmlotRIrt/9hZq+tp32sl7LbIHR8EQXm56eQtp2MlK0V6E69xl8C/hBDLhBDDI9uOlFJuAYj8PiLO94gLuxu0qCCfy0/Mt5Nn10NfpWplJP14Ul7KqnlZkRo7WVpUkG8bn62V1lL12H65XX1pVQBERn4SUUIuqJWh6bTC1M8qa0XjJl7j3l9K2Re4ALhVCGG+PNQEIcRwIcRSIcTS7du3x9kMa5xu0Plrt3vWpxcV5DPq/G6EsqJNYSjLuRqRGXaji1jceo05oWxfmnGJuxqiTp6kPhJxg1Vf/RKUl5tu3rL+cDarUatQxBJXzF1KuTnye5sQ4i2gH7BVCNFOSrlFCNEO2GZx7BRgCmhqmXjaYYfTEnAvQ+x6+8ZaQRO3182Eqt3oIvb4vNyQZRbJFgZ1TlW4hjunr2Dphp310h3k5YQsPWo4qHCxa7OdXNNP2uEgQx1BrQxVK0wVmYxvKaQQogWQJaX8OfL3XOBh4Bxgh2FCtbWU8o9250plgWwvckij7MyNNM1tPm+rc7XKDbEvXBu9mClLgCCqOIjTKtTfntIpysCbpRGw6oMf/EhMlaRPofBOoqSQRwJvCS1jYhNgmpTyAyHE58B0IcT1wA/AFXG8RyDYKVbMPPvsLFFvwU/scNzNZNtD76x2pbawGl1IWb9At26Qs4WgRsooDfmd01eYtum1xRujjLv+3sYEYFb9tMLugWnnhbfKDbFnX3XUg0VNDCoUwePbuEspvwdOMNm+A817zwisht5m24wG2UmaVlJabhk+iTV+Vm2wW80Zm6lS32a1r1m/jRp0L2EHJ/231bXRvfN0WRikUDRkGnX6AR0rz97O4JjlkoktSm2FUwUoHackZrGjAN2bj8WuEIcfHb6T/ttpniMR2n+FQhFN5hfrSAElpeXMXFYeZdgFRCXpsgtNuA1BuNHhG99n6MkdTfex2u4Xp5CUku0pFKlHee4+MPNcJZqsUscqNJGXE3Jt5IzhGisP3jgK0OPqry3eSI2UZAvB0JM71lPLxIub1ZLKO1coUosy7j5wM5lqFZoYO7iHp/cyxsbdVPV5pKhX4MY8FrOqS341/gqFIjEo4+4Dt54rBKeRTjvNtQuNv0KhSB2NOuWvX9zq1xsqKv2sQpEeqEpMAZN2XnSSSaeEWgqFwhxl3H3SWCcMS0rLybKQXKr0swpF+qCkkArX6OEoM8OuVpkqFOmFMu4K11ilG/ZSh1WhUCQHZdwVrrGKqddKqQy7QpFmKOOucE06Fa9QKBT2KOOucE26Fa9QKBTWKLWMwjWNXQKqUGQSyrgrPNFYJaAKRaahwjIKhULRAFHGXaFQKBogyrgrFApFA0QZd4VCoWiAKOOuUCgUDZC0SPkrhNgObDBsagP8mKLmJJvG0lfVz4ZHY+lrOvezs5SyrdkLaWHcYxFCLLXKUdzQaCx9Vf1seDSWvmZqP1VYRqFQKBogyrgrFApFAyRdjfuUVDcgiTSWvqp+NjwaS18zsp9pGXNXKBQKRXykq+euUCgUijhQxl2hUCgaICkz7kKI1kKIuUKIbyK/W1ns96IQYpsQ4ks/x6caD/38lRBinRDiWyHEaMP2sUKIciHE8sjPhclrvTNW7Ta8LoQQf468vlII0dftselGnH0tE0KsinyGS5Pbcm+46Gd3IcRnQoj9Qoi7vBybTsTZz/T/PKWUKfkBHgNGR/4eDfzJYr8zgb7Al36OT/WPm3YC2cB3wNFAU2AFcHzktbHAXanuh0XfLNtt2OdC4H1AAKcAi90em04/8fQ18loZ0CbV/Qion0cAJwHjjd/NTPpM4+lnpnyeqQzLXAK8HPn7ZaDIbCcp5QJgp9/j0wA37ewHfCul/F5KeQB4PXJcuuOm3ZcAr0iNRUCeEKKdy2PTiXj6mkk49lNKuU1K+TkQ9npsGhFPPzOCVBr3I6WUWwAiv49I8vHJwk0784GNhv83Rbbp3BYZ5r+YZuEnp3bb7ePm2HQinr4CSOBfQohlQojhCWtl/MTzuWTSZxpvW9P+80xoJSYhxDzgKJOX7kvk+yabAPopTLbpGtXngHGR/8cBTwD/47WNCcKu3U77uDk2nYinrwD9pZSbhRBHAHOFEGsjo9J0I57PJZM+03jbmvafZ0KNu5TyXKvXhBBbhRDtpJRbIkPXbR5PH+/xgRFAPzcBHQ3/dwA2R8691XCu54F3g2l1IFi228U+TV0cm07E01eklPrvbUKIt9DCAmllDCK46Wcijk02cbU1Ez7PVIZlZgHDIn8PA95O8vHJwk07PweOFUJ0FUI0Ba6KHEdMzPZS4EuT41OFZbsNzAKujShJTgF2R8JTbo5NJ3z3VQjRQghxKIAQogUwiPT6HI3E87lk0mfqu60Z83mmcLb6cOBD4JvI79aR7e2B9wz7vQZsQZvU2ARcb3d8uv146OeFwNdoM/j3Gbb/E1gFrET78rVLdZ9i+lev3cD/Av8b+VsAf428vgoodOpzuv747SuaImNF5Gd1uvfVRT+PityLPwEVkb8Py7TP1G8/M+XzVOkHFAqFogGiVqgqFApFA0QZd4VCoWiAKOOuUCgUDRBl3BUKhaIBooy7QqFQNECUcVcoFIoGiDLuCoVC0QD5/7bFEOaRiBcrAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import sklearn\n",
    "from sklearn import datasets\n",
    "\n",
    "# load data\n",
    "d = datasets.load_diabetes()\n",
    "\n",
    "X = d.data[:, 2]\n",
    "Y = d.target\n",
    "\n",
    "N = X.shape[0]\n",
    "\n",
    "#key approach\n",
    "n_epoch = 500          # epoch size\n",
    "a, b = 1, 1             # initial parameters\n",
    "epsilon = 0.01         # learning rate\n",
    "\n",
    "for i in range(n_epoch):\n",
    "    for j in range(N):\n",
    "        #iteration to find the best a b\n",
    "        #相当于求出sum号里面每一项，然后依次加到ab上，就能得到a1 b1\n",
    "        a = a + epsilon*2*(Y[j] - a*X[j] - b)*X[j]\n",
    "        b = b + epsilon*2*(Y[j] - a*X[j] - b)\n",
    "\n",
    "    L = 0\n",
    "    for j in range(N):\n",
    "        #求出在最佳ab下的损失函数函数值\n",
    "        L = L + (Y[j]-a*X[j]-b)**2\n",
    "    #print(\"epoch %4d: loss = %f, a = %f, b = %f\" % (i, L, a, b))\n",
    "    \n",
    "x_min = np.min(X)\n",
    "x_max = np.max(X)\n",
    "y_min = a * x_min + b\n",
    "y_max = a * x_max + b\n",
    "\n",
    "plt.scatter(X, Y, label='original data')\n",
    "plt.plot([x_min, x_max], [y_min, y_max], 'r', label='model')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "98682e17",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "/* Put everything inside the global mpl namespace */\n",
       "/* global mpl */\n",
       "window.mpl = {};\n",
       "\n",
       "mpl.get_websocket_type = function () {\n",
       "    if (typeof WebSocket !== 'undefined') {\n",
       "        return WebSocket;\n",
       "    } else if (typeof MozWebSocket !== 'undefined') {\n",
       "        return MozWebSocket;\n",
       "    } else {\n",
       "        alert(\n",
       "            'Your browser does not have WebSocket support. ' +\n",
       "                'Please try Chrome, Safari or Firefox ≥ 6. ' +\n",
       "                'Firefox 4 and 5 are also supported but you ' +\n",
       "                'have to enable WebSockets in about:config.'\n",
       "        );\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure = function (figure_id, websocket, ondownload, parent_element) {\n",
       "    this.id = figure_id;\n",
       "\n",
       "    this.ws = websocket;\n",
       "\n",
       "    this.supports_binary = this.ws.binaryType !== undefined;\n",
       "\n",
       "    if (!this.supports_binary) {\n",
       "        var warnings = document.getElementById('mpl-warnings');\n",
       "        if (warnings) {\n",
       "            warnings.style.display = 'block';\n",
       "            warnings.textContent =\n",
       "                'This browser does not support binary websocket messages. ' +\n",
       "                'Performance may be slow.';\n",
       "        }\n",
       "    }\n",
       "\n",
       "    this.imageObj = new Image();\n",
       "\n",
       "    this.context = undefined;\n",
       "    this.message = undefined;\n",
       "    this.canvas = undefined;\n",
       "    this.rubberband_canvas = undefined;\n",
       "    this.rubberband_context = undefined;\n",
       "    this.format_dropdown = undefined;\n",
       "\n",
       "    this.image_mode = 'full';\n",
       "\n",
       "    this.root = document.createElement('div');\n",
       "    this.root.setAttribute('style', 'display: inline-block');\n",
       "    this._root_extra_style(this.root);\n",
       "\n",
       "    parent_element.appendChild(this.root);\n",
       "\n",
       "    this._init_header(this);\n",
       "    this._init_canvas(this);\n",
       "    this._init_toolbar(this);\n",
       "\n",
       "    var fig = this;\n",
       "\n",
       "    this.waiting = false;\n",
       "\n",
       "    this.ws.onopen = function () {\n",
       "        fig.send_message('supports_binary', { value: fig.supports_binary });\n",
       "        fig.send_message('send_image_mode', {});\n",
       "        if (fig.ratio !== 1) {\n",
       "            fig.send_message('set_dpi_ratio', { dpi_ratio: fig.ratio });\n",
       "        }\n",
       "        fig.send_message('refresh', {});\n",
       "    };\n",
       "\n",
       "    this.imageObj.onload = function () {\n",
       "        if (fig.image_mode === 'full') {\n",
       "            // Full images could contain transparency (where diff images\n",
       "            // almost always do), so we need to clear the canvas so that\n",
       "            // there is no ghosting.\n",
       "            fig.context.clearRect(0, 0, fig.canvas.width, fig.canvas.height);\n",
       "        }\n",
       "        fig.context.drawImage(fig.imageObj, 0, 0);\n",
       "    };\n",
       "\n",
       "    this.imageObj.onunload = function () {\n",
       "        fig.ws.close();\n",
       "    };\n",
       "\n",
       "    this.ws.onmessage = this._make_on_message_function(this);\n",
       "\n",
       "    this.ondownload = ondownload;\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._init_header = function () {\n",
       "    var titlebar = document.createElement('div');\n",
       "    titlebar.classList =\n",
       "        'ui-dialog-titlebar ui-widget-header ui-corner-all ui-helper-clearfix';\n",
       "    var titletext = document.createElement('div');\n",
       "    titletext.classList = 'ui-dialog-title';\n",
       "    titletext.setAttribute(\n",
       "        'style',\n",
       "        'width: 100%; text-align: center; padding: 3px;'\n",
       "    );\n",
       "    titlebar.appendChild(titletext);\n",
       "    this.root.appendChild(titlebar);\n",
       "    this.header = titletext;\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._canvas_extra_style = function (_canvas_div) {};\n",
       "\n",
       "mpl.figure.prototype._root_extra_style = function (_canvas_div) {};\n",
       "\n",
       "mpl.figure.prototype._init_canvas = function () {\n",
       "    var fig = this;\n",
       "\n",
       "    var canvas_div = (this.canvas_div = document.createElement('div'));\n",
       "    canvas_div.setAttribute(\n",
       "        'style',\n",
       "        'border: 1px solid #ddd;' +\n",
       "            'box-sizing: content-box;' +\n",
       "            'clear: both;' +\n",
       "            'min-height: 1px;' +\n",
       "            'min-width: 1px;' +\n",
       "            'outline: 0;' +\n",
       "            'overflow: hidden;' +\n",
       "            'position: relative;' +\n",
       "            'resize: both;'\n",
       "    );\n",
       "\n",
       "    function on_keyboard_event_closure(name) {\n",
       "        return function (event) {\n",
       "            return fig.key_event(event, name);\n",
       "        };\n",
       "    }\n",
       "\n",
       "    canvas_div.addEventListener(\n",
       "        'keydown',\n",
       "        on_keyboard_event_closure('key_press')\n",
       "    );\n",
       "    canvas_div.addEventListener(\n",
       "        'keyup',\n",
       "        on_keyboard_event_closure('key_release')\n",
       "    );\n",
       "\n",
       "    this._canvas_extra_style(canvas_div);\n",
       "    this.root.appendChild(canvas_div);\n",
       "\n",
       "    var canvas = (this.canvas = document.createElement('canvas'));\n",
       "    canvas.classList.add('mpl-canvas');\n",
       "    canvas.setAttribute('style', 'box-sizing: content-box;');\n",
       "\n",
       "    this.context = canvas.getContext('2d');\n",
       "\n",
       "    var backingStore =\n",
       "        this.context.backingStorePixelRatio ||\n",
       "        this.context.webkitBackingStorePixelRatio ||\n",
       "        this.context.mozBackingStorePixelRatio ||\n",
       "        this.context.msBackingStorePixelRatio ||\n",
       "        this.context.oBackingStorePixelRatio ||\n",
       "        this.context.backingStorePixelRatio ||\n",
       "        1;\n",
       "\n",
       "    this.ratio = (window.devicePixelRatio || 1) / backingStore;\n",
       "\n",
       "    var rubberband_canvas = (this.rubberband_canvas = document.createElement(\n",
       "        'canvas'\n",
       "    ));\n",
       "    rubberband_canvas.setAttribute(\n",
       "        'style',\n",
       "        'box-sizing: content-box; position: absolute; left: 0; top: 0; z-index: 1;'\n",
       "    );\n",
       "\n",
       "    // Apply a ponyfill if ResizeObserver is not implemented by browser.\n",
       "    if (this.ResizeObserver === undefined) {\n",
       "        if (window.ResizeObserver !== undefined) {\n",
       "            this.ResizeObserver = window.ResizeObserver;\n",
       "        } else {\n",
       "            var obs = _JSXTOOLS_RESIZE_OBSERVER({});\n",
       "            this.ResizeObserver = obs.ResizeObserver;\n",
       "        }\n",
       "    }\n",
       "\n",
       "    this.resizeObserverInstance = new this.ResizeObserver(function (entries) {\n",
       "        var nentries = entries.length;\n",
       "        for (var i = 0; i < nentries; i++) {\n",
       "            var entry = entries[i];\n",
       "            var width, height;\n",
       "            if (entry.contentBoxSize) {\n",
       "                if (entry.contentBoxSize instanceof Array) {\n",
       "                    // Chrome 84 implements new version of spec.\n",
       "                    width = entry.contentBoxSize[0].inlineSize;\n",
       "                    height = entry.contentBoxSize[0].blockSize;\n",
       "                } else {\n",
       "                    // Firefox implements old version of spec.\n",
       "                    width = entry.contentBoxSize.inlineSize;\n",
       "                    height = entry.contentBoxSize.blockSize;\n",
       "                }\n",
       "            } else {\n",
       "                // Chrome <84 implements even older version of spec.\n",
       "                width = entry.contentRect.width;\n",
       "                height = entry.contentRect.height;\n",
       "            }\n",
       "\n",
       "            // Keep the size of the canvas and rubber band canvas in sync with\n",
       "            // the canvas container.\n",
       "            if (entry.devicePixelContentBoxSize) {\n",
       "                // Chrome 84 implements new version of spec.\n",
       "                canvas.setAttribute(\n",
       "                    'width',\n",
       "                    entry.devicePixelContentBoxSize[0].inlineSize\n",
       "                );\n",
       "                canvas.setAttribute(\n",
       "                    'height',\n",
       "                    entry.devicePixelContentBoxSize[0].blockSize\n",
       "                );\n",
       "            } else {\n",
       "                canvas.setAttribute('width', width * fig.ratio);\n",
       "                canvas.setAttribute('height', height * fig.ratio);\n",
       "            }\n",
       "            canvas.setAttribute(\n",
       "                'style',\n",
       "                'width: ' + width + 'px; height: ' + height + 'px;'\n",
       "            );\n",
       "\n",
       "            rubberband_canvas.setAttribute('width', width);\n",
       "            rubberband_canvas.setAttribute('height', height);\n",
       "\n",
       "            // And update the size in Python. We ignore the initial 0/0 size\n",
       "            // that occurs as the element is placed into the DOM, which should\n",
       "            // otherwise not happen due to the minimum size styling.\n",
       "            if (fig.ws.readyState == 1 && width != 0 && height != 0) {\n",
       "                fig.request_resize(width, height);\n",
       "            }\n",
       "        }\n",
       "    });\n",
       "    this.resizeObserverInstance.observe(canvas_div);\n",
       "\n",
       "    function on_mouse_event_closure(name) {\n",
       "        return function (event) {\n",
       "            return fig.mouse_event(event, name);\n",
       "        };\n",
       "    }\n",
       "\n",
       "    rubberband_canvas.addEventListener(\n",
       "        'mousedown',\n",
       "        on_mouse_event_closure('button_press')\n",
       "    );\n",
       "    rubberband_canvas.addEventListener(\n",
       "        'mouseup',\n",
       "        on_mouse_event_closure('button_release')\n",
       "    );\n",
       "    // Throttle sequential mouse events to 1 every 20ms.\n",
       "    rubberband_canvas.addEventListener(\n",
       "        'mousemove',\n",
       "        on_mouse_event_closure('motion_notify')\n",
       "    );\n",
       "\n",
       "    rubberband_canvas.addEventListener(\n",
       "        'mouseenter',\n",
       "        on_mouse_event_closure('figure_enter')\n",
       "    );\n",
       "    rubberband_canvas.addEventListener(\n",
       "        'mouseleave',\n",
       "        on_mouse_event_closure('figure_leave')\n",
       "    );\n",
       "\n",
       "    canvas_div.addEventListener('wheel', function (event) {\n",
       "        if (event.deltaY < 0) {\n",
       "            event.step = 1;\n",
       "        } else {\n",
       "            event.step = -1;\n",
       "        }\n",
       "        on_mouse_event_closure('scroll')(event);\n",
       "    });\n",
       "\n",
       "    canvas_div.appendChild(canvas);\n",
       "    canvas_div.appendChild(rubberband_canvas);\n",
       "\n",
       "    this.rubberband_context = rubberband_canvas.getContext('2d');\n",
       "    this.rubberband_context.strokeStyle = '#000000';\n",
       "\n",
       "    this._resize_canvas = function (width, height, forward) {\n",
       "        if (forward) {\n",
       "            canvas_div.style.width = width + 'px';\n",
       "            canvas_div.style.height = height + 'px';\n",
       "        }\n",
       "    };\n",
       "\n",
       "    // Disable right mouse context menu.\n",
       "    this.rubberband_canvas.addEventListener('contextmenu', function (_e) {\n",
       "        event.preventDefault();\n",
       "        return false;\n",
       "    });\n",
       "\n",
       "    function set_focus() {\n",
       "        canvas.focus();\n",
       "        canvas_div.focus();\n",
       "    }\n",
       "\n",
       "    window.setTimeout(set_focus, 100);\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._init_toolbar = function () {\n",
       "    var fig = this;\n",
       "\n",
       "    var toolbar = document.createElement('div');\n",
       "    toolbar.classList = 'mpl-toolbar';\n",
       "    this.root.appendChild(toolbar);\n",
       "\n",
       "    function on_click_closure(name) {\n",
       "        return function (_event) {\n",
       "            return fig.toolbar_button_onclick(name);\n",
       "        };\n",
       "    }\n",
       "\n",
       "    function on_mouseover_closure(tooltip) {\n",
       "        return function (event) {\n",
       "            if (!event.currentTarget.disabled) {\n",
       "                return fig.toolbar_button_onmouseover(tooltip);\n",
       "            }\n",
       "        };\n",
       "    }\n",
       "\n",
       "    fig.buttons = {};\n",
       "    var buttonGroup = document.createElement('div');\n",
       "    buttonGroup.classList = 'mpl-button-group';\n",
       "    for (var toolbar_ind in mpl.toolbar_items) {\n",
       "        var name = mpl.toolbar_items[toolbar_ind][0];\n",
       "        var tooltip = mpl.toolbar_items[toolbar_ind][1];\n",
       "        var image = mpl.toolbar_items[toolbar_ind][2];\n",
       "        var method_name = mpl.toolbar_items[toolbar_ind][3];\n",
       "\n",
       "        if (!name) {\n",
       "            /* Instead of a spacer, we start a new button group. */\n",
       "            if (buttonGroup.hasChildNodes()) {\n",
       "                toolbar.appendChild(buttonGroup);\n",
       "            }\n",
       "            buttonGroup = document.createElement('div');\n",
       "            buttonGroup.classList = 'mpl-button-group';\n",
       "            continue;\n",
       "        }\n",
       "\n",
       "        var button = (fig.buttons[name] = document.createElement('button'));\n",
       "        button.classList = 'mpl-widget';\n",
       "        button.setAttribute('role', 'button');\n",
       "        button.setAttribute('aria-disabled', 'false');\n",
       "        button.addEventListener('click', on_click_closure(method_name));\n",
       "        button.addEventListener('mouseover', on_mouseover_closure(tooltip));\n",
       "\n",
       "        var icon_img = document.createElement('img');\n",
       "        icon_img.src = '_images/' + image + '.png';\n",
       "        icon_img.srcset = '_images/' + image + '_large.png 2x';\n",
       "        icon_img.alt = tooltip;\n",
       "        button.appendChild(icon_img);\n",
       "\n",
       "        buttonGroup.appendChild(button);\n",
       "    }\n",
       "\n",
       "    if (buttonGroup.hasChildNodes()) {\n",
       "        toolbar.appendChild(buttonGroup);\n",
       "    }\n",
       "\n",
       "    var fmt_picker = document.createElement('select');\n",
       "    fmt_picker.classList = 'mpl-widget';\n",
       "    toolbar.appendChild(fmt_picker);\n",
       "    this.format_dropdown = fmt_picker;\n",
       "\n",
       "    for (var ind in mpl.extensions) {\n",
       "        var fmt = mpl.extensions[ind];\n",
       "        var option = document.createElement('option');\n",
       "        option.selected = fmt === mpl.default_extension;\n",
       "        option.innerHTML = fmt;\n",
       "        fmt_picker.appendChild(option);\n",
       "    }\n",
       "\n",
       "    var status_bar = document.createElement('span');\n",
       "    status_bar.classList = 'mpl-message';\n",
       "    toolbar.appendChild(status_bar);\n",
       "    this.message = status_bar;\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.request_resize = function (x_pixels, y_pixels) {\n",
       "    // Request matplotlib to resize the figure. Matplotlib will then trigger a resize in the client,\n",
       "    // which will in turn request a refresh of the image.\n",
       "    this.send_message('resize', { width: x_pixels, height: y_pixels });\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.send_message = function (type, properties) {\n",
       "    properties['type'] = type;\n",
       "    properties['figure_id'] = this.id;\n",
       "    this.ws.send(JSON.stringify(properties));\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.send_draw_message = function () {\n",
       "    if (!this.waiting) {\n",
       "        this.waiting = true;\n",
       "        this.ws.send(JSON.stringify({ type: 'draw', figure_id: this.id }));\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_save = function (fig, _msg) {\n",
       "    var format_dropdown = fig.format_dropdown;\n",
       "    var format = format_dropdown.options[format_dropdown.selectedIndex].value;\n",
       "    fig.ondownload(fig, format);\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_resize = function (fig, msg) {\n",
       "    var size = msg['size'];\n",
       "    if (size[0] !== fig.canvas.width || size[1] !== fig.canvas.height) {\n",
       "        fig._resize_canvas(size[0], size[1], msg['forward']);\n",
       "        fig.send_message('refresh', {});\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_rubberband = function (fig, msg) {\n",
       "    var x0 = msg['x0'] / fig.ratio;\n",
       "    var y0 = (fig.canvas.height - msg['y0']) / fig.ratio;\n",
       "    var x1 = msg['x1'] / fig.ratio;\n",
       "    var y1 = (fig.canvas.height - msg['y1']) / fig.ratio;\n",
       "    x0 = Math.floor(x0) + 0.5;\n",
       "    y0 = Math.floor(y0) + 0.5;\n",
       "    x1 = Math.floor(x1) + 0.5;\n",
       "    y1 = Math.floor(y1) + 0.5;\n",
       "    var min_x = Math.min(x0, x1);\n",
       "    var min_y = Math.min(y0, y1);\n",
       "    var width = Math.abs(x1 - x0);\n",
       "    var height = Math.abs(y1 - y0);\n",
       "\n",
       "    fig.rubberband_context.clearRect(\n",
       "        0,\n",
       "        0,\n",
       "        fig.canvas.width / fig.ratio,\n",
       "        fig.canvas.height / fig.ratio\n",
       "    );\n",
       "\n",
       "    fig.rubberband_context.strokeRect(min_x, min_y, width, height);\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_figure_label = function (fig, msg) {\n",
       "    // Updates the figure title.\n",
       "    fig.header.textContent = msg['label'];\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_cursor = function (fig, msg) {\n",
       "    var cursor = msg['cursor'];\n",
       "    switch (cursor) {\n",
       "        case 0:\n",
       "            cursor = 'pointer';\n",
       "            break;\n",
       "        case 1:\n",
       "            cursor = 'default';\n",
       "            break;\n",
       "        case 2:\n",
       "            cursor = 'crosshair';\n",
       "            break;\n",
       "        case 3:\n",
       "            cursor = 'move';\n",
       "            break;\n",
       "    }\n",
       "    fig.rubberband_canvas.style.cursor = cursor;\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_message = function (fig, msg) {\n",
       "    fig.message.textContent = msg['message'];\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_draw = function (fig, _msg) {\n",
       "    // Request the server to send over a new figure.\n",
       "    fig.send_draw_message();\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_image_mode = function (fig, msg) {\n",
       "    fig.image_mode = msg['mode'];\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_history_buttons = function (fig, msg) {\n",
       "    for (var key in msg) {\n",
       "        if (!(key in fig.buttons)) {\n",
       "            continue;\n",
       "        }\n",
       "        fig.buttons[key].disabled = !msg[key];\n",
       "        fig.buttons[key].setAttribute('aria-disabled', !msg[key]);\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_navigate_mode = function (fig, msg) {\n",
       "    if (msg['mode'] === 'PAN') {\n",
       "        fig.buttons['Pan'].classList.add('active');\n",
       "        fig.buttons['Zoom'].classList.remove('active');\n",
       "    } else if (msg['mode'] === 'ZOOM') {\n",
       "        fig.buttons['Pan'].classList.remove('active');\n",
       "        fig.buttons['Zoom'].classList.add('active');\n",
       "    } else {\n",
       "        fig.buttons['Pan'].classList.remove('active');\n",
       "        fig.buttons['Zoom'].classList.remove('active');\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.updated_canvas_event = function () {\n",
       "    // Called whenever the canvas gets updated.\n",
       "    this.send_message('ack', {});\n",
       "};\n",
       "\n",
       "// A function to construct a web socket function for onmessage handling.\n",
       "// Called in the figure constructor.\n",
       "mpl.figure.prototype._make_on_message_function = function (fig) {\n",
       "    return function socket_on_message(evt) {\n",
       "        if (evt.data instanceof Blob) {\n",
       "            /* FIXME: We get \"Resource interpreted as Image but\n",
       "             * transferred with MIME type text/plain:\" errors on\n",
       "             * Chrome.  But how to set the MIME type?  It doesn't seem\n",
       "             * to be part of the websocket stream */\n",
       "            evt.data.type = 'image/png';\n",
       "\n",
       "            /* Free the memory for the previous frames */\n",
       "            if (fig.imageObj.src) {\n",
       "                (window.URL || window.webkitURL).revokeObjectURL(\n",
       "                    fig.imageObj.src\n",
       "                );\n",
       "            }\n",
       "\n",
       "            fig.imageObj.src = (window.URL || window.webkitURL).createObjectURL(\n",
       "                evt.data\n",
       "            );\n",
       "            fig.updated_canvas_event();\n",
       "            fig.waiting = false;\n",
       "            return;\n",
       "        } else if (\n",
       "            typeof evt.data === 'string' &&\n",
       "            evt.data.slice(0, 21) === 'data:image/png;base64'\n",
       "        ) {\n",
       "            fig.imageObj.src = evt.data;\n",
       "            fig.updated_canvas_event();\n",
       "            fig.waiting = false;\n",
       "            return;\n",
       "        }\n",
       "\n",
       "        var msg = JSON.parse(evt.data);\n",
       "        var msg_type = msg['type'];\n",
       "\n",
       "        // Call the  \"handle_{type}\" callback, which takes\n",
       "        // the figure and JSON message as its only arguments.\n",
       "        try {\n",
       "            var callback = fig['handle_' + msg_type];\n",
       "        } catch (e) {\n",
       "            console.log(\n",
       "                \"No handler for the '\" + msg_type + \"' message type: \",\n",
       "                msg\n",
       "            );\n",
       "            return;\n",
       "        }\n",
       "\n",
       "        if (callback) {\n",
       "            try {\n",
       "                // console.log(\"Handling '\" + msg_type + \"' message: \", msg);\n",
       "                callback(fig, msg);\n",
       "            } catch (e) {\n",
       "                console.log(\n",
       "                    \"Exception inside the 'handler_\" + msg_type + \"' callback:\",\n",
       "                    e,\n",
       "                    e.stack,\n",
       "                    msg\n",
       "                );\n",
       "            }\n",
       "        }\n",
       "    };\n",
       "};\n",
       "\n",
       "// from http://stackoverflow.com/questions/1114465/getting-mouse-location-in-canvas\n",
       "mpl.findpos = function (e) {\n",
       "    //this section is from http://www.quirksmode.org/js/events_properties.html\n",
       "    var targ;\n",
       "    if (!e) {\n",
       "        e = window.event;\n",
       "    }\n",
       "    if (e.target) {\n",
       "        targ = e.target;\n",
       "    } else if (e.srcElement) {\n",
       "        targ = e.srcElement;\n",
       "    }\n",
       "    if (targ.nodeType === 3) {\n",
       "        // defeat Safari bug\n",
       "        targ = targ.parentNode;\n",
       "    }\n",
       "\n",
       "    // pageX,Y are the mouse positions relative to the document\n",
       "    var boundingRect = targ.getBoundingClientRect();\n",
       "    var x = e.pageX - (boundingRect.left + document.body.scrollLeft);\n",
       "    var y = e.pageY - (boundingRect.top + document.body.scrollTop);\n",
       "\n",
       "    return { x: x, y: y };\n",
       "};\n",
       "\n",
       "/*\n",
       " * return a copy of an object with only non-object keys\n",
       " * we need this to avoid circular references\n",
       " * http://stackoverflow.com/a/24161582/3208463\n",
       " */\n",
       "function simpleKeys(original) {\n",
       "    return Object.keys(original).reduce(function (obj, key) {\n",
       "        if (typeof original[key] !== 'object') {\n",
       "            obj[key] = original[key];\n",
       "        }\n",
       "        return obj;\n",
       "    }, {});\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.mouse_event = function (event, name) {\n",
       "    var canvas_pos = mpl.findpos(event);\n",
       "\n",
       "    if (name === 'button_press') {\n",
       "        this.canvas.focus();\n",
       "        this.canvas_div.focus();\n",
       "    }\n",
       "\n",
       "    var x = canvas_pos.x * this.ratio;\n",
       "    var y = canvas_pos.y * this.ratio;\n",
       "\n",
       "    this.send_message(name, {\n",
       "        x: x,\n",
       "        y: y,\n",
       "        button: event.button,\n",
       "        step: event.step,\n",
       "        guiEvent: simpleKeys(event),\n",
       "    });\n",
       "\n",
       "    /* This prevents the web browser from automatically changing to\n",
       "     * the text insertion cursor when the button is pressed.  We want\n",
       "     * to control all of the cursor setting manually through the\n",
       "     * 'cursor' event from matplotlib */\n",
       "    event.preventDefault();\n",
       "    return false;\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._key_event_extra = function (_event, _name) {\n",
       "    // Handle any extra behaviour associated with a key event\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.key_event = function (event, name) {\n",
       "    // Prevent repeat events\n",
       "    if (name === 'key_press') {\n",
       "        if (event.which === this._key) {\n",
       "            return;\n",
       "        } else {\n",
       "            this._key = event.which;\n",
       "        }\n",
       "    }\n",
       "    if (name === 'key_release') {\n",
       "        this._key = null;\n",
       "    }\n",
       "\n",
       "    var value = '';\n",
       "    if (event.ctrlKey && event.which !== 17) {\n",
       "        value += 'ctrl+';\n",
       "    }\n",
       "    if (event.altKey && event.which !== 18) {\n",
       "        value += 'alt+';\n",
       "    }\n",
       "    if (event.shiftKey && event.which !== 16) {\n",
       "        value += 'shift+';\n",
       "    }\n",
       "\n",
       "    value += 'k';\n",
       "    value += event.which.toString();\n",
       "\n",
       "    this._key_event_extra(event, name);\n",
       "\n",
       "    this.send_message(name, { key: value, guiEvent: simpleKeys(event) });\n",
       "    return false;\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.toolbar_button_onclick = function (name) {\n",
       "    if (name === 'download') {\n",
       "        this.handle_save(this, null);\n",
       "    } else {\n",
       "        this.send_message('toolbar_button', { name: name });\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.toolbar_button_onmouseover = function (tooltip) {\n",
       "    this.message.textContent = tooltip;\n",
       "};\n",
       "\n",
       "///////////////// REMAINING CONTENT GENERATED BY embed_js.py /////////////////\n",
       "// prettier-ignore\n",
       "var _JSXTOOLS_RESIZE_OBSERVER=function(A){var t,i=new WeakMap,n=new WeakMap,a=new WeakMap,r=new WeakMap,o=new Set;function s(e){if(!(this instanceof s))throw new TypeError(\"Constructor requires 'new' operator\");i.set(this,e)}function h(){throw new TypeError(\"Function is not a constructor\")}function c(e,t,i,n){e=0 in arguments?Number(arguments[0]):0,t=1 in arguments?Number(arguments[1]):0,i=2 in arguments?Number(arguments[2]):0,n=3 in arguments?Number(arguments[3]):0,this.right=(this.x=this.left=e)+(this.width=i),this.bottom=(this.y=this.top=t)+(this.height=n),Object.freeze(this)}function d(){t=requestAnimationFrame(d);var s=new WeakMap,p=new Set;o.forEach((function(t){r.get(t).forEach((function(i){var r=t instanceof window.SVGElement,o=a.get(t),d=r?0:parseFloat(o.paddingTop),f=r?0:parseFloat(o.paddingRight),l=r?0:parseFloat(o.paddingBottom),u=r?0:parseFloat(o.paddingLeft),g=r?0:parseFloat(o.borderTopWidth),m=r?0:parseFloat(o.borderRightWidth),w=r?0:parseFloat(o.borderBottomWidth),b=u+f,F=d+l,v=(r?0:parseFloat(o.borderLeftWidth))+m,W=g+w,y=r?0:t.offsetHeight-W-t.clientHeight,E=r?0:t.offsetWidth-v-t.clientWidth,R=b+v,z=F+W,M=r?t.width:parseFloat(o.width)-R-E,O=r?t.height:parseFloat(o.height)-z-y;if(n.has(t)){var k=n.get(t);if(k[0]===M&&k[1]===O)return}n.set(t,[M,O]);var S=Object.create(h.prototype);S.target=t,S.contentRect=new c(u,d,M,O),s.has(i)||(s.set(i,[]),p.add(i)),s.get(i).push(S)}))})),p.forEach((function(e){i.get(e).call(e,s.get(e),e)}))}return s.prototype.observe=function(i){if(i instanceof window.Element){r.has(i)||(r.set(i,new Set),o.add(i),a.set(i,window.getComputedStyle(i)));var n=r.get(i);n.has(this)||n.add(this),cancelAnimationFrame(t),t=requestAnimationFrame(d)}},s.prototype.unobserve=function(i){if(i instanceof window.Element&&r.has(i)){var n=r.get(i);n.has(this)&&(n.delete(this),n.size||(r.delete(i),o.delete(i))),n.size||r.delete(i),o.size||cancelAnimationFrame(t)}},A.DOMRectReadOnly=c,A.ResizeObserver=s,A.ResizeObserverEntry=h,A}; // eslint-disable-line\n",
       "mpl.toolbar_items = [[\"Home\", \"Reset original view\", \"fa fa-home icon-home\", \"home\"], [\"Back\", \"Back to previous view\", \"fa fa-arrow-left icon-arrow-left\", \"back\"], [\"Forward\", \"Forward to next view\", \"fa fa-arrow-right icon-arrow-right\", \"forward\"], [\"\", \"\", \"\", \"\"], [\"Pan\", \"Left button pans, Right button zooms\\nx/y fixes axis, CTRL fixes aspect\", \"fa fa-arrows icon-move\", \"pan\"], [\"Zoom\", \"Zoom to rectangle\\nx/y fixes axis, CTRL fixes aspect\", \"fa fa-square-o icon-check-empty\", \"zoom\"], [\"\", \"\", \"\", \"\"], [\"Download\", \"Download plot\", \"fa fa-floppy-o icon-save\", \"download\"]];\n",
       "\n",
       "mpl.extensions = [\"eps\", \"jpeg\", \"pdf\", \"png\", \"ps\", \"raw\", \"svg\", \"tif\"];\n",
       "\n",
       "mpl.default_extension = \"png\";/* global mpl */\n",
       "\n",
       "var comm_websocket_adapter = function (comm) {\n",
       "    // Create a \"websocket\"-like object which calls the given IPython comm\n",
       "    // object with the appropriate methods. Currently this is a non binary\n",
       "    // socket, so there is still some room for performance tuning.\n",
       "    var ws = {};\n",
       "\n",
       "    ws.close = function () {\n",
       "        comm.close();\n",
       "    };\n",
       "    ws.send = function (m) {\n",
       "        //console.log('sending', m);\n",
       "        comm.send(m);\n",
       "    };\n",
       "    // Register the callback with on_msg.\n",
       "    comm.on_msg(function (msg) {\n",
       "        //console.log('receiving', msg['content']['data'], msg);\n",
       "        // Pass the mpl event to the overridden (by mpl) onmessage function.\n",
       "        ws.onmessage(msg['content']['data']);\n",
       "    });\n",
       "    return ws;\n",
       "};\n",
       "\n",
       "mpl.mpl_figure_comm = function (comm, msg) {\n",
       "    // This is the function which gets called when the mpl process\n",
       "    // starts-up an IPython Comm through the \"matplotlib\" channel.\n",
       "\n",
       "    var id = msg.content.data.id;\n",
       "    // Get hold of the div created by the display call when the Comm\n",
       "    // socket was opened in Python.\n",
       "    var element = document.getElementById(id);\n",
       "    var ws_proxy = comm_websocket_adapter(comm);\n",
       "\n",
       "    function ondownload(figure, _format) {\n",
       "        window.open(figure.canvas.toDataURL());\n",
       "    }\n",
       "\n",
       "    var fig = new mpl.figure(id, ws_proxy, ondownload, element);\n",
       "\n",
       "    // Call onopen now - mpl needs it, as it is assuming we've passed it a real\n",
       "    // web socket which is closed, not our websocket->open comm proxy.\n",
       "    ws_proxy.onopen();\n",
       "\n",
       "    fig.parent_element = element;\n",
       "    fig.cell_info = mpl.find_output_cell(\"<div id='\" + id + \"'></div>\");\n",
       "    if (!fig.cell_info) {\n",
       "        console.error('Failed to find cell for figure', id, fig);\n",
       "        return;\n",
       "    }\n",
       "    fig.cell_info[0].output_area.element.on(\n",
       "        'cleared',\n",
       "        { fig: fig },\n",
       "        fig._remove_fig_handler\n",
       "    );\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_close = function (fig, msg) {\n",
       "    var width = fig.canvas.width / fig.ratio;\n",
       "    fig.cell_info[0].output_area.element.off(\n",
       "        'cleared',\n",
       "        fig._remove_fig_handler\n",
       "    );\n",
       "    fig.resizeObserverInstance.unobserve(fig.canvas_div);\n",
       "\n",
       "    // Update the output cell to use the data from the current canvas.\n",
       "    fig.push_to_output();\n",
       "    var dataURL = fig.canvas.toDataURL();\n",
       "    // Re-enable the keyboard manager in IPython - without this line, in FF,\n",
       "    // the notebook keyboard shortcuts fail.\n",
       "    IPython.keyboard_manager.enable();\n",
       "    fig.parent_element.innerHTML =\n",
       "        '<img src=\"' + dataURL + '\" width=\"' + width + '\">';\n",
       "    fig.close_ws(fig, msg);\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.close_ws = function (fig, msg) {\n",
       "    fig.send_message('closing', msg);\n",
       "    // fig.ws.close()\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.push_to_output = function (_remove_interactive) {\n",
       "    // Turn the data on the canvas into data in the output cell.\n",
       "    var width = this.canvas.width / this.ratio;\n",
       "    var dataURL = this.canvas.toDataURL();\n",
       "    this.cell_info[1]['text/html'] =\n",
       "        '<img src=\"' + dataURL + '\" width=\"' + width + '\">';\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.updated_canvas_event = function () {\n",
       "    // Tell IPython that the notebook contents must change.\n",
       "    IPython.notebook.set_dirty(true);\n",
       "    this.send_message('ack', {});\n",
       "    var fig = this;\n",
       "    // Wait a second, then push the new image to the DOM so\n",
       "    // that it is saved nicely (might be nice to debounce this).\n",
       "    setTimeout(function () {\n",
       "        fig.push_to_output();\n",
       "    }, 1000);\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._init_toolbar = function () {\n",
       "    var fig = this;\n",
       "\n",
       "    var toolbar = document.createElement('div');\n",
       "    toolbar.classList = 'btn-toolbar';\n",
       "    this.root.appendChild(toolbar);\n",
       "\n",
       "    function on_click_closure(name) {\n",
       "        return function (_event) {\n",
       "            return fig.toolbar_button_onclick(name);\n",
       "        };\n",
       "    }\n",
       "\n",
       "    function on_mouseover_closure(tooltip) {\n",
       "        return function (event) {\n",
       "            if (!event.currentTarget.disabled) {\n",
       "                return fig.toolbar_button_onmouseover(tooltip);\n",
       "            }\n",
       "        };\n",
       "    }\n",
       "\n",
       "    fig.buttons = {};\n",
       "    var buttonGroup = document.createElement('div');\n",
       "    buttonGroup.classList = 'btn-group';\n",
       "    var button;\n",
       "    for (var toolbar_ind in mpl.toolbar_items) {\n",
       "        var name = mpl.toolbar_items[toolbar_ind][0];\n",
       "        var tooltip = mpl.toolbar_items[toolbar_ind][1];\n",
       "        var image = mpl.toolbar_items[toolbar_ind][2];\n",
       "        var method_name = mpl.toolbar_items[toolbar_ind][3];\n",
       "\n",
       "        if (!name) {\n",
       "            /* Instead of a spacer, we start a new button group. */\n",
       "            if (buttonGroup.hasChildNodes()) {\n",
       "                toolbar.appendChild(buttonGroup);\n",
       "            }\n",
       "            buttonGroup = document.createElement('div');\n",
       "            buttonGroup.classList = 'btn-group';\n",
       "            continue;\n",
       "        }\n",
       "\n",
       "        button = fig.buttons[name] = document.createElement('button');\n",
       "        button.classList = 'btn btn-default';\n",
       "        button.href = '#';\n",
       "        button.title = name;\n",
       "        button.innerHTML = '<i class=\"fa ' + image + ' fa-lg\"></i>';\n",
       "        button.addEventListener('click', on_click_closure(method_name));\n",
       "        button.addEventListener('mouseover', on_mouseover_closure(tooltip));\n",
       "        buttonGroup.appendChild(button);\n",
       "    }\n",
       "\n",
       "    if (buttonGroup.hasChildNodes()) {\n",
       "        toolbar.appendChild(buttonGroup);\n",
       "    }\n",
       "\n",
       "    // Add the status bar.\n",
       "    var status_bar = document.createElement('span');\n",
       "    status_bar.classList = 'mpl-message pull-right';\n",
       "    toolbar.appendChild(status_bar);\n",
       "    this.message = status_bar;\n",
       "\n",
       "    // Add the close button to the window.\n",
       "    var buttongrp = document.createElement('div');\n",
       "    buttongrp.classList = 'btn-group inline pull-right';\n",
       "    button = document.createElement('button');\n",
       "    button.classList = 'btn btn-mini btn-primary';\n",
       "    button.href = '#';\n",
       "    button.title = 'Stop Interaction';\n",
       "    button.innerHTML = '<i class=\"fa fa-power-off icon-remove icon-large\"></i>';\n",
       "    button.addEventListener('click', function (_evt) {\n",
       "        fig.handle_close(fig, {});\n",
       "    });\n",
       "    button.addEventListener(\n",
       "        'mouseover',\n",
       "        on_mouseover_closure('Stop Interaction')\n",
       "    );\n",
       "    buttongrp.appendChild(button);\n",
       "    var titlebar = this.root.querySelector('.ui-dialog-titlebar');\n",
       "    titlebar.insertBefore(buttongrp, titlebar.firstChild);\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._remove_fig_handler = function (event) {\n",
       "    var fig = event.data.fig;\n",
       "    if (event.target !== this) {\n",
       "        // Ignore bubbled events from children.\n",
       "        return;\n",
       "    }\n",
       "    fig.close_ws(fig, {});\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._root_extra_style = function (el) {\n",
       "    el.style.boxSizing = 'content-box'; // override notebook setting of border-box.\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._canvas_extra_style = function (el) {\n",
       "    // this is important to make the div 'focusable\n",
       "    el.setAttribute('tabindex', 0);\n",
       "    // reach out to IPython and tell the keyboard manager to turn it's self\n",
       "    // off when our div gets focus\n",
       "\n",
       "    // location in version 3\n",
       "    if (IPython.notebook.keyboard_manager) {\n",
       "        IPython.notebook.keyboard_manager.register_events(el);\n",
       "    } else {\n",
       "        // location in version 2\n",
       "        IPython.keyboard_manager.register_events(el);\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._key_event_extra = function (event, _name) {\n",
       "    var manager = IPython.notebook.keyboard_manager;\n",
       "    if (!manager) {\n",
       "        manager = IPython.keyboard_manager;\n",
       "    }\n",
       "\n",
       "    // Check for shift+enter\n",
       "    if (event.shiftKey && event.which === 13) {\n",
       "        this.canvas_div.blur();\n",
       "        // select the cell after this one\n",
       "        var index = IPython.notebook.find_cell_index(this.cell_info[0]);\n",
       "        IPython.notebook.select(index + 1);\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_save = function (fig, _msg) {\n",
       "    fig.ondownload(fig, null);\n",
       "};\n",
       "\n",
       "mpl.find_output_cell = function (html_output) {\n",
       "    // Return the cell and output element which can be found *uniquely* in the notebook.\n",
       "    // Note - this is a bit hacky, but it is done because the \"notebook_saving.Notebook\"\n",
       "    // IPython event is triggered only after the cells have been serialised, which for\n",
       "    // our purposes (turning an active figure into a static one), is too late.\n",
       "    var cells = IPython.notebook.get_cells();\n",
       "    var ncells = cells.length;\n",
       "    for (var i = 0; i < ncells; i++) {\n",
       "        var cell = cells[i];\n",
       "        if (cell.cell_type === 'code') {\n",
       "            for (var j = 0; j < cell.output_area.outputs.length; j++) {\n",
       "                var data = cell.output_area.outputs[j];\n",
       "                if (data.data) {\n",
       "                    // IPython >= 3 moved mimebundle to data attribute of output\n",
       "                    data = data.data;\n",
       "                }\n",
       "                if (data['text/html'] === html_output) {\n",
       "                    return [cell, data, j];\n",
       "                }\n",
       "            }\n",
       "        }\n",
       "    }\n",
       "};\n",
       "\n",
       "// Register the function which deals with the matplotlib target/channel.\n",
       "// The kernel may be null if the page has been refreshed.\n",
       "if (IPython.notebook.kernel !== null) {\n",
       "    IPython.notebook.kernel.comm_manager.register_target(\n",
       "        'matplotlib',\n",
       "        mpl.mpl_figure_comm\n",
       "    );\n",
       "}\n"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<img src=\"\" width=\"475.45453514934593\">"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#animation visualization\n",
    "%matplotlib nbagg\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "import matplotlib.animation as animation\n",
    "\n",
    "n_epoch = 3000          # epoch size\n",
    "a, b = 1, 1             # initial parameters\n",
    "epsilon = 0.001         # learning rate\n",
    "\n",
    "fig = plt.figure()\n",
    "imgs = []\n",
    "\n",
    "for i in range(n_epoch):\n",
    "    for j in range(N):\n",
    "        a = a + epsilon*2*(Y[j] - a*X[j] - b)*X[j]\n",
    "        b = b + epsilon*2*(Y[j] - a*X[j] - b)\n",
    "\n",
    "    L = 0\n",
    "    for j in range(N):\n",
    "        L = L + (Y[j]-a*X[j]-b)**2\n",
    "    #print(\"epoch %4d: loss = %f, a = %f, b = %f\" % (i, L, a, b))\n",
    "    \n",
    "    if i % 50 == 0:\n",
    "        x_min = np.min(X)\n",
    "        x_max = np.max(X)\n",
    "        y_min = a * x_min + b\n",
    "        y_max = a * x_max + b\n",
    "\n",
    "        img = plt.scatter(X, Y, label='original data')\n",
    "        img = plt.plot([x_min, x_max], [y_min, y_max], 'r', label='model')\n",
    "        imgs.append(img)\n",
    "        \n",
    "ani = animation.ArtistAnimation(fig, imgs)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "52d4d3c2",
   "metadata": {},
   "source": [
    "### 使用sklearn实现线性拟合"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "98ca0b41",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X:  (442, 1)\n",
      "Y:  (442,)\n",
      "a = 949.435260, b = 152.133484\n"
     ]
    }
   ],
   "source": [
    "#sklearn里面的linear regression对函数做了一些优化\n",
    "from sklearn import linear_model\n",
    "from sklearn import datasets\n",
    "import numpy as np\n",
    "\n",
    "# load data\n",
    "d = datasets.load_diabetes()\n",
    "\n",
    "X = d.data[:, np.newaxis, 2]\n",
    "Y = d.target\n",
    "print(\"X: \", X.shape)\n",
    "print(\"Y: \", Y.shape)\n",
    "\n",
    "# create regression model\n",
    "regr = linear_model.LinearRegression()\n",
    "regr.fit(X, Y)\n",
    "\n",
    "a, b = regr.coef_, regr.intercept_\n",
    "print(\"a = %f, b = %f\" % (a, b))\n",
    "\n",
    "x_min = np.min(X)\n",
    "x_max = np.max(X)\n",
    "y_min = a * x_min + b\n",
    "y_max = a * x_max + b\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
